Electronic device and method for controlling operation of accessory-mountable robot

ABSTRACT

An electronic device such as an accessory-mountable robot is provided. The electronic device changes functional properties thereof in accordance with a mounted accessory. In an embodiment, the electronic device detects mounting of at least one accessory and identifies accessory characteristics associated with the at least one accessory. Then, the electronic device determines properties of the electronic device associated with the at least one accessory, based on the accessory characteristics, and changes the properties of the electronic device, based on the determined properties. Also, the electronic device outputs at least one of a visual element, an auditory element, or a tactile element associated with the at least one accessory, based on the changed properties.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of a Korean patent application number 10-2018-0164896, filed onDec. 19, 2018, in the Korean Intellectual Property Office, thedisclosure of which is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to an accessory-mountable robot and, moreparticularly, to an electronic device and control method for changing afunctional property of a robot in accordance with a mounted accessory.

2. Description of Related Art

With the growths of related technologies, common robots (or robotdevices) as well as industrial robots are actively studied anddeveloped, and people's interest in common robots is also increasing. Arobot is a mechanical artifact that has a certain outward form and iscapable of performing mechanical movements and actions. For example, arobot may mean a device having the ability to work on its own.

Recently, intelligent robots that recognize their environments andthereby control and determine their own behaviors have been introduced.The intelligent robots can provide various kinds of services to usersthrough active execution suitable for environments. For example, theintelligent robots may be classified into pet robots, silver robots,secretary robots, guide robots, and the like according to their specificproperties (or roles, services, etc.), and may provide particularservices based on the specific properties to users.

These various kinds of robots always have unvaried their shapes. Inaddition, when operating (or providing a service), each robot merelyrepeats a given motion, action, sound, etc. in accordance with itsinherent property. Therefore, the user using the robot may easily feeltired of the robot and have a limitation on the use of the robot.

Accordingly, there is an increasing need for differentiated robots tomeet various demands of users.

The above information is presented as background information only, andto assist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providean accessory-mountable robot (or robot device or electronic device) thatallows various accessories to be selectively mounted thereon. Inaddition, various embodiments of the disclosure provide an electronicdevice and control method for changing a functional property of theaccessory-mountable robot in accordance with a mounted accessory.

Another aspect of the disclosure is to provide an electronic device andmethod for controlling an operation of an accessory-mountable robot bychanging a functional property or state (e.g., default property,function, and/or UX attribute) of the robot, depending onaccessory-related characteristics (e.g., types, positions, and/or numberof mounted accessories).

Another aspect of the disclosure is to provide an electronic device andmethod for controlling an operation of an accessory-mountable robot bydetermining accessory characteristics through a combination of thetypes, positions, and/or number of accessories mounted on the robot and,based on the determined accessory characteristics, changing a functionalproperty of the robot.

Another aspect of the disclosure is to provide an electronic device andmethod for controlling an operation of an accessory-mountable robot bychanging a functional property of the robot through a combination of afirst characteristic, such as the types, positions, or number ofaccessories mounted on the robot, and a second characteristic, such asan operating time or place of the robot.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, an electronic device isprovided. The electronic device includes a housing, at least one sensorcircuit disposed in the housing and configured to detect mounting of atleast one accessory, at least one communication circuit disposed in thehousing and configured to communicate with the at least one accessory, adisplay device disposed in the housing and configured to display visualinformation associated with the at least one accessory, and at least oneprocessor disposed in the housing and operatively connected to thesensor circuit, the communication circuit, and the display device.

In accordance with another aspect of the disclosure, the at least oneprocessor may be configured to detect the mounting of the at least oneaccessory, identify accessory characteristics associated with the atleast one accessory, determine properties of the electronic deviceassociated with the at least one accessory, based on the accessorycharacteristics, change the properties of the electronic device, basedon the determined properties, and output as feedback at least one of avisual element, an auditory element, or a tactile element associatedwith the at least one accessory, based on the changed properties.

In accordance with another aspect of the disclosure, an operating methodof an electronic device is provided. The operating method includesdetecting mounting of at least one accessory, identifying accessorycharacteristics associated with the at least one accessory, determiningproperties of the electronic device associated with the at least oneaccessory, based on the accessory characteristics, changing theproperties of the electronic device, based on the determined properties,and outputting as feedback at least one element of a visual element, anauditory element, or a tactile element associated with the at least oneaccessory, based on the changed properties.

According to various embodiments of the disclosure, a non-transitorycomputer-readable recording medium having recorded thereon a program forexecuting the above method on a processor may be provided.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment according to an embodiment of the disclosure;

FIG. 2 is a diagram illustrating an integrated intelligence systemaccording to an embodiment of the disclosure;

FIG. 3 is a block diagram illustrating an electronic device of anintegrated intelligence system according to an embodiment of thedisclosure;

FIG. 4 is a diagram illustrating a process of collecting a current stateat an intelligent service module according to an embodiment of thedisclosure;

FIG. 5 is a diagram illustrating an example of an electronic deviceaccording to an embodiment of the disclosure;

FIG. 6 is a diagram illustrating mountable positions of an accessory ona robot according to an embodiment of the disclosure;

FIG. 7 is a block diagram illustrating a robot and an accessoryaccording to an embodiment of the disclosure;

FIG. 8 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure;

FIG. 9 is a diagram illustrating a method for identifyingcharacteristics of an accessory at a robot according to an embodiment ofthe disclosure;

FIG. 10 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure;

FIG. 11 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure;

FIG. 12 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure;

FIG. 13 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure;

FIG. 14 is a diagram illustrating examples of an accessory mounted on arobot according to an embodiment of the disclosure;

FIG. 15 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 16 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 17 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure;

FIG. 18 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure;

FIG. 19 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure;

FIG. 20 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure;

FIG. 21 is a diagram illustrating examples of an accessory mounted on arobot according to an embodiment of the disclosure;

FIG. 22 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 23 is a diagram illustrating examples of an accessory mounted on arobot according to an embodiment of the disclosure;

FIG. 24 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 25 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 26 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 27 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure;

FIG. 28 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure; and

FIGS. 29, 30, 31, and 32 are diagrams illustrating examples of anaccessory combination mounted on a robot and a related operationaccording to various embodiments of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

FIG. 1 is a block diagram illustrating an electronic device in a networkenvironment according to an embodiment of the disclosure.

Referring to FIG. 1 , an electronic device 101 in a network environment100 may communicate with an electronic device 102 via a first network198 (e.g., a short-range wireless communication network), or anelectronic device 104 or a server 108 via a second network 199 (e.g., along-range wireless communication network). According to an embodiment,the electronic device 101 may communicate with the electronic device 104via the server 108. According to an embodiment, the electronic device101 may include a processor 120, a memory 130, an input device 150, asound output device 155, a display device 160, an audio module 170, asensor module 176, an interface 177, a haptic module 179, a cameramodule 180, a power management module 188, a battery 189, acommunication module 190, a subscriber identification module (SIM) 196,or an antenna module 197. In some embodiments, at least one (e.g., thedisplay device 160 or the camera module 180) of the components may beomitted from the electronic device 101, or one or more other componentsmay be added in the electronic device 101. In some embodiments, some ofthe components may be implemented as single integrated circuitry. Forexample, the sensor module 176 (e.g., a fingerprint sensor, an irissensor, or an illuminance sensor) may be implemented as embedded in thedisplay device 160 (e.g., a display).

The processor 120 may execute, for example, software (e.g., a program140) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 101 coupled with theprocessor 120, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 120 may load a command or data received fromanother component (e.g., the sensor module 176 or the communicationmodule 190) in volatile memory 132, process the command or the datastored in the volatile memory 132, and store resulting data innon-volatile memory 134. According to an embodiment, the processor 120may include a main processor 121 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 123 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor121. Additionally or alternatively, the auxiliary processor 123 may beadapted to consume less power than the main processor 121, or to bespecific to a specified function. The auxiliary processor 123 may beimplemented as separate from, or as part of the main processor 121.

The auxiliary processor 123 may control at least some of functions orstates related to at least one component (e.g., the display device 160,the sensor module 176, or the communication module 190) among thecomponents of the electronic device 101, instead of the main processor121 while the main processor 121 is in an inactive (e.g., sleep) state,or together with the main processor 121 while the main processor 121 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 123 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 180 or the communication module 190)functionally related to the auxiliary processor 123.

The memory 130 may store various data used by at least one component(e.g., the processor 120 or the sensor module 176) of the electronicdevice 101. The various data may include, for example, software (e.g.,the program 140) and input data or output data for a command relatedthereto. The memory 130 may include the volatile memory 132 or thenon-volatile memory 134. The non-volatile memory 134 may include aninternal memory 136 and/or an external memory 138.

The program 140 may be stored in the memory 130 as software, and mayinclude, for example, an operating system (OS) 142, middleware 144, oran application 146.

The input device 150 may receive a command or data to be used by othercomponents (e.g., the processor 120) of the electronic device 101, fromthe outside (e.g., a user) of the electronic device 101. The inputdevice 150 may include, for example, a microphone, a mouse, a keyboard,or a digital pen (e.g., a stylus pen).

The sound output device 155 may output sound signals to the outside ofthe electronic device 101. The sound output device 155 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for incoming calls. According to an embodiment, the receivermay be implemented as separate from, or as part of the speaker.

The display device 160 may visually provide information to the outside(e.g., a user) of the electronic device 101. The display device 160 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 160 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 170 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 170 may obtainthe sound via the input device 150, or output the sound via the soundoutput device 155 or a headphone of an external electronic device (e.g.,an electronic device 102) directly (e.g., over wires) or wirelesslycoupled with the electronic device 101.

The sensor module 176 may detect an operational state (e.g., power ortemperature) of the electronic device 101 or an environmental state(e.g., a state of a user) external to the electronic device 101, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 176 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 177 may support one or more specified protocols to be usedfor the electronic device 101 to be coupled with the external electronicdevice (e.g., the electronic device 102) directly (e.g., over wires) orwirelessly. According to an embodiment, the interface 177 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 178 may include a connector via which theelectronic device 101 may be physically connected with the externalelectronic device (e.g., the electronic device 102). According to anembodiment, the connecting terminal 178 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 179 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 180 may capture a still image or moving images.According to an embodiment, the camera module 180 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to theelectronic device 101. According to one embodiment, the power managementmodule 188 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 189 may supply power to at least one component of theelectronic device 101. According to an embodiment, the battery 189 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 190 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 101 and the external electronic device (e.g., theelectronic device 102, the electronic device 104, or the server 108) andperforming communication via the established communication channel. Thecommunication module 190 may include one or more communicationprocessors that are operable independently from the processor 120 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 190 may include a wireless communication module192 (e.g., a cellular communication module, a short-range wirelesscommunication module, or a global navigation satellite system (GNSS)communication module) or a wired communication module 194 (e.g., a localarea network (LAN) communication module or a power line communication(PLC) module). A corresponding one of these communication modules maycommunicate with the external electronic device via the first network198 (e.g., a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or the second network 199 (e.g., a long-range communication network,such as a cellular network, the Internet, or a computer network (e.g., aLAN or a wide area network (WAN)). These various types of communicationmodules may be implemented as a single component (e.g., a single chip),or may be implemented as multi components (e.g., multi chips) separatefrom each other. The wireless communication module 192 may identify andauthenticate the electronic device 101 in a communication network, suchas the first network 198 or the second network 199, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 196.

The antenna module 197 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 101. According to an embodiment, the antenna module197 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., a printed circuit board (PCB)). According to an embodiment, theantenna module 197 may include a plurality of antennas. In such a case,at least one antenna appropriate for a communication scheme used in thecommunication network, such as the first network 198 or the secondnetwork 199, may be selected, for example, by the communication module190 (e.g., the wireless communication module 192) from the plurality ofantennas. The signal or the power may then be transmitted or receivedbetween the communication module 190 and the external electronic devicevia the selected at least one antenna. According to an embodiment,another component (e.g., a radio frequency integrated circuit (RFIC))other than the radiating element may be additionally formed as part ofthe antenna module 197.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 101 and the external electronicdevice 104 via the server 108 coupled with the second network 199. Eachof the electronic devices 102 and 104 may be a device of a same type as,or a different type, from the electronic device 101.

According to an embodiment, all or some of operations to be executed atthe electronic device 101 may be executed at one or more of the externalelectronic devices 102, 104, or 108. For example, if the electronicdevice 101 should perform a function or a service automatically, or inresponse to a request from a user or another device, the electronicdevice 101, instead of, or in addition to, executing the function or theservice, may request the one or more external electronic devices toperform at least part of the function or the service. The one or moreexternal electronic devices receiving the request may perform the atleast part of the function or the service requested, or an additionalfunction or an additional service related to the request, and transferan outcome of the performing to the electronic device 101. Theelectronic device 101 may provide the outcome, with or without furtherprocessing of the outcome, as at least part of a reply to the request.To that end, a cloud computing, distributed computing, or client-servercomputing technology may be used, for example.

The electronic device 101 according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smartphone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the disclosure andthe terms used therein are not intended to limit the technologicalfeatures set forth herein to particular embodiments and include variouschanges, equivalents, or replacements for a corresponding embodiment.With regard to the description of the drawings, similar referencenumerals may be used to refer to similar or related elements. It is tobe understood that a singular form of a noun corresponding to an itemmay include one or more of the things, unless the relevant contextclearly indicates otherwise.

As used herein, each of such phrases as “A or B”, “at least one of A andB”, “at least one of A or B”, “A, B, or C”, “at least one of A, B, andC”, and “at least one of A, B, or C” may include any one of, or allpossible combinations of the items enumerated together in acorresponding one of the phrases. As used herein, such terms as “1^(st)”and “2^(nd)”, or “first” and “second” may be used to simply distinguisha corresponding component from another, and does not limit thecomponents in other aspect (e.g., importance or order). It is to beunderstood that if an element (e.g., a first element) is referred to,with or without the term “operatively” or “communicatively”, as “coupledwith”, “coupled to”, “connected with”, or “connected to” another element(e.g., a second element), it means that the element may be coupled withthe other element directly (e.g., over wires), wirelessly, or via athird element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic”, “logic block”, “part”, or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 140) including one or more instructions that arestored in a storage medium (e.g., internal memory 136 or external memory138) that is readable by a machine (e.g., the electronic device 101).For example, a processor (e.g., the processor 120) of the machine (e.g.,the electronic device 101) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a complier or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., PlayStore™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

At the outset, an integrated intelligence system to which embodiments ofthe disclosure can be applied will be described hereinafter.

FIG. 2 is a diagram illustrating an integrated intelligence systemaccording to an embodiment of the disclosure.

Referring to FIG. 2 , an integrated intelligence system 20 may includean embodied electronic device 101, an intelligent server 310, apersonalization information server 330, and a proposal server 350.

The electronic device 101 may provide a particular service to a userthrough an application (or an application program, hereinafter referredto as ‘app’, e.g., an alarm app, a message app, a photo (or gallery)app, etc.) installed therein. For example, the electronic device 101 mayexecute and operate a desired app through an intelligent app (or acontext awareness app or a voice recognition app) installed therein.Also, the electronic device 101 may receive a user input for executingand operating a desired app through the intelligent app. This user inputmay be received, for example, via an accessory, a physical button, atouch pad, a voice input, a remote input, or the like. According toembodiments, the electronic device 101 may be a robot, a mobile phone, asmart phone, a laptop computer, a notebook computer, a personal digitalassistant (PDA), or any other terminal device (or electronic device)capable of connecting to the Internet.

According to an embodiment, the electronic device 101 may receive auser's speech as a user input. Upon receiving the user's speech, theelectronic device 101 may generate a command to operate a particularapp, based on the user's speech, and operate the app in accordance withthe command. According to another embodiment, the electronic device 101may detect mounting of an accessory, recognize it as a user input,generate a command to operate a particular app, and operate the app inaccordance with the command.

The intelligent server 310 may receive a user voice input from theelectronic device 101 through a communication network and change it intorelevant data (e.g., text data) for context awareness. In an embodiment,the intelligent server 310 may receive information (e.g., accessorycharacteristics) about the mounted accessory from the electronic device101 through the communication network and change the receivedinformation into relevant data for context awareness. In anotherembodiment, the intelligent server 310 may generate (or select) a pathrule, based on text data. The path rule may include information aboutactions (or operations) for performing a function of an app orinformation about parameters required for executing the action. Inaddition, the path rule may include a sequence of actions or operationsof the app. The electronic device 101 may receive the path rule, selectan app according to the received path rule, and execute actions oroperations defined in the path rule in the selected app.

In the disclosure, the term “path rule” generally refers to, but is notlimited to, a sequence of states for the electronic device 101 toperform a task requested by a user or a context-aware task according tothe mounting of an accessory. That is, the path rule may includeinformation about the sequence of states. The task may be, for example,any action that an intelligent app can provide. The task may includeexpressing (or feeding back) characteristics corresponding to theaccessory, generating a schedule, sending a photo to a desired party, orproviding weather information. The electronic device 101 may perform thetask according to the sequence of at least one state (e.g., an operatingstate of the electronic device 101).

According to an embodiment, the path rule may be provided or generatedby an artificial intelligent (AI) system. The AI system may be arule-based system, a neural network-based system (e.g., a feedforwardneural network (FNN), a recurrent neural network (RNN)), a combinationof such systems, or any other AI system. In an embodiment, the path rulemay be selected from a set of predefined path rules or generated in realtime in response to a user request. For example, the AI system mayselect at least one of a plurality of predefined path rules or generatethe path rule dynamically (or in real time). In addition, the electronicdevice 101 may use a hybrid system to provide the path rule.

According to an embodiment, the electronic device 101 may execute anaction and display a screen corresponding to the state thereof on adisplay (e.g., the display device 160 in FIG. 1 ). However, even thoughexecuting an action, the electronic device 101 may not display a resultof executing the action on the display. In another embodiment, theelectronic device 101 may execute a plurality of actions and displayonly a part of results of the plurality of actions on the display. Instill another embodiment, the electronic device 101 may display only aresult of executing the last action on the display. In yet anotherembodiment, the electronic device 101 may display on the display only aresult of executing an action by receiving a user input.

The personalization information server 330 may include a database (notshown) that stores various kinds of information about the electronicdevice 101, accessories mountable on the electronic device 101, and/or auser of the electronic device 101. For example, the personalizationinformation server 330 may receive user information (e.g., accessoryinformation, context information, app execution information, etc.) fromthe electronic device 101 and store it in the database. The intelligentserver 310 may receive various kinds of information from thepersonalization information server 330 through the communication networkand use the received information in generating the path rule withrespect to a user input and/or mounting an accessory on the electronicdevice 101. In an embodiment, the electronic device 101 may receivevarious kinds of information from the personalization information server330 through the communication network and use the received informationin managing the database.

The proposal server 350 may include a database (not shown) that storesinformation about functions to be provided to the electronic device 101.For example, the proposal server 350 may receive user information (ordevice information) of the electronic device 101 from thepersonalization information server 330 and correlate the userinformation with functions that the user can use through the electronicdevice 101. The electronic device 101 may receive information about afunction, to be provided, from the proposal server 350 through thecommunication network and offer the received information to the user.

FIG. 3 is a block diagram illustrating an electronic device of anintegrated intelligence system according to an embodiment of thedisclosure.

Referring to FIG. 3 , the electronic device 101 may include an inputmodule 210 (e.g., the input device 150 in FIG. 1 ), an expression device220 (e.g., the display device 160 in FIG. 1 ), a speaker 230 (e.g., thesound output device 155 in FIG. 1 ), a memory 240 (e.g., the memory 130in FIG. 1 ), and a processor 250 (e.g., the processor 120 in FIG. 1 ).The electronic device 101 may further include a housing (not shown), andthe above-listed components of the electronic device 101 may be disposedin or on the housing. The electronic device 101 may further include acommunication circuit (e.g., the communication module 190 in FIG. 1 )disposed in the housing and is capable of transmitting/receiving data(or information) to/from an external server (e.g., the intelligentserver 310) through the communication circuit.

According to an embodiment, the input module 210 may receive a userinput from a user. For example, the input module 210 may receive theuser input from a connected external device (e.g., a keyboard, aheadset, a tablet PC, a projector). In another example, the input module210 may include a touch screen (or a touch screen display) combined witha display in the expression device 220 for expressing an emotion (e.g.,a facial expression). In still another example, the input module 210 mayinclude a hardware key (or a physical key) disposed on the electronicdevice 101 (or on the housing of the electronic device 101). In yetanother example, the input module 210 may include at least one distancesensor (e.g., an infrared sensor, an ultrasonic sensor) disposed in theelectronic device 101 (or in the housing of the electronic device 101).For example, the input module 210 may include one or more sensorscapable of receiving a user's contact (e.g., a touch, a hug, a stroke)as a gesture signal.

According to an embodiment, the input module 210 may include amicrophone capable of receiving a user's speech as a voice signal. Forexample, the input module 210 may include a speech input system andreceive a user's speech as a voice signal through the speech inputsystem. The microphone may be exposed through a portion (e.g., a firstportion) of the housing.

According to an embodiment, the expression device 220 may include adevice capable of expressing an emotion and a facial expression. Forexample, the expression device 220 may include a device capable ofexpressing a facial expression corresponding to an emotion to beexpressed by the electronic device 101 (e.g., a robot). In anembodiment, the expression device 220 may include various devicescapable of expressing a facial expression such as an eyebrow, a pupil, anose, a cheek, and a lip, as well as a display that generates a graphicof a facial expression. In an embodiment, the expression device 220 mayinclude a device that implements a facial movement such as moving eyes(e.g., wink) or moving lips (e.g., biting or pursing lips) and therebyexpresses a happy, sad, pleasant, smiling, angry, embarrassing, ordissatisfied facial expression.

According to an embodiment, when the expression device 220 is a display,it can display an image, a video, and/or an execution screen (e.g., agraphic element) of an application. For example, the expression device220 (e.g., the display) may display a graphic user interface (GUI) of anapp. In an embodiment, the expression device 220 may be exposed througha portion (e.g., a second portion) of the housing. In an embodiment, theexpression device 220 may display various expressions as images oranimations by interacting with (or including) a device (e.g., a unityengine) capable of expressing a face (or facial expression graphic)corresponding to an accessory.

According to an embodiment, the speaker 230 may output various soundsignals (e.g., voice, music, sound effect, natural sound, etc.). Forexample, the speaker 230 may output a voice signal or a sound effectsignal, generated in the electronic device 101, to the outside. In anembodiment, the speaker 230 may be exposed through a portion (e.g., athird portion) of the housing.

According to an embodiment, the memory 240 may store a plurality of apps(or application programs) (e.g., a first app 241 and a second app 243).Each of the plurality of apps 241 and 243 may be, for example, a programfor performing a function corresponding to mounting of an accessory(e.g., a function of an emotion expression such as a facial expressionchange, a sound change, or a motion change, a function of extendingexisting functions, a function of interworking existing functions) or aprogram for performing a function corresponding to a user input. In anembodiment, the memory 240 may store an intelligent agent 245, anexecution manager module 247, and/or an intelligent service module 249,each of which may be, for example, a framework (or an applicationframework) for processing a received user input (e.g., a user speech,mounting of an accessory).

According to an embodiment, the memory 240 may include a database thatstores information necessary for recognizing a user input (e.g., a userspeech, mounting of an accessory). For example, the memory 240 mayinclude a log database for storing log information. For example, thememory 240 may include a persona database for storing information abouta user and/or an accessory.

According to an embodiment, the plurality of apps 241 and 243 stored inthe memory 240 may be loaded and operated by, for example, the executionmanager module 247. The plurality of apps 241 and 243 may includeexecution service modules 241 a and 243 a to perform a function. In anembodiment, the plurality of apps 241 and 243 may execute a plurality ofactions (e.g., the sequence of states) 241 b and 243 b through theexecution service modules 241 a and 243 a to perform a function. Thatis, the execution service modules 241 a and 243 a may be activated bythe execution manager module 247 and execute the plurality of actions241 b and 243 b.

According to an embodiment, when the actions 241 b and 243 b of the apps241 and 243 are executed, an execution status screen resulting from theexecution of the actions 241 b and 243 b may be displayed on theexpression device 220 (e.g., a display). The execution status screen maybe, for example, a screen that shows a status where the actions 241 band 243 b are completed. In another example, the execution status screenmay be a screen that shows a status (also referred to as partiallanding) where the execution of the actions 241 b and 243 b is stopped(e.g., when a parameter required for the actions 241 b and 243 b is notinput).

According to an embodiment, the execution service modules 241 a or 243 amay execute the actions 241 b and 243 b in accordance with the pathrule. For example, the execution service modules 241 a and 243 a may beactivated by the execution manager module 247, receive an executionrequest from the execution manager module 247 in accordance with thepath rule, and perform the functions of the apps 241 and 243 byexecuting the actions 241 b and 243 b in response to the executionrequest. When the execution of the actions 241 b and 243 b is completed,the execution service modules 241 a and 243 a may deliver completioninformation to the execution manager module 247.

According to an embodiment, when the plurality of actions 241 b and 243b are executed in the apps 241 and 243, these actions 241 b and 243 bmay be sequentially executed. That is, when the execution of one action(e.g., action 1 of the first app 241, action 1 of the second app 243) iscompleted, the execution service modules 241 a and 243 a may open thenext action (e.g., action 2 of the first app 241, action 2 of the secondapp 243) and send the completion information to the execution managermodule 247. Here, opening any action may refer to changing (orswitching) such an action to an executable state or preparing to executesuch an action. In other words, if any action is not open, that actioncannot be executed. When the completion information is received, theexecution manager module 247 may deliver an execution request for thenext action (e.g., action 2 of the first app 241, action 2 of the secondapp 243) to the execution service modules 241 a and 243 a. In anembodiment, the plurality of apps 241 and 243 as well may besequentially executed. For example, when the execution of the lastaction of the first app 241 (e.g., action 3 of the first app 241) iscompleted and thereby the completion information is received, theexecution manager module 247 may send an execution request for the firstaction of the second app 243 (e.g., action 1 of the second app 243) tothe execution service module 243 a.

According to an embodiment, when the plurality of actions 241 b and 243b are executed in the apps 241 and 243, the screen resulting from theexecution of each of the actions 241 b and 243 b may be displayed on theexpression device 220. In an embodiment, only a part of screensresulting from the execution of the actions 241 b and 243 b may bedisplayed on the expression device 220.

According to an embodiment, the memory 240 may store an intelligent app(e.g., a voice recognition app, a context awareness app) linked to theintelligent agent 245. This app linked to the intelligent agent 245 mayreceive and process a user's speech as a voice signal. In an embodiment,the app linked to the intelligent agent 245 may receive and process asignal of detecting (or connecting) the mounting of an accessory. In anembodiment, the app linked to the intelligent agent 245 may be operatedin response to a specific input entered through the input module 210(e.g., an input through a hardware key, an input through a touch screen,a specific voice input, an input through a sensor).

According to an embodiment, the intelligent agent 245, the executionmanager module 247, and/or the intelligent service module 249 stored inthe memory 240 may be executed by the processor 250. That is, thefunctions of the intelligent agent 245, the execution manager module247, and/or the intelligent service module 249 may be implemented by theprocessor 250. In various embodiments disclosed herein, the functions ofthe intelligent agent 245, the execution manager module 247, and theintelligent service module 249 will be described as operations of theprocessor 250 (e.g., the processor 120 in FIG. 1 ). In an embodiment,the intelligent agent 245, the execution manager module 247, and/or theintelligent service module 249 stored in the memory 240 may beimplemented in hardware as well as software.

According to an embodiment, the processor 250 may control overalloperations of the electronic device 101. For example, the processor 250may detect the mounting of an accessory, identify the characteristics ofthe accessory (e.g., an accessory type, an accessory mounting position,the number of mounted accessories) in response to the detection of theaccessory, and output a corresponding feedback based on thecharacteristics of the accessory. The processor 250 may control theinput module 210 to receive a user input. The processor 250 may controlthe expression device 220 to display an image. The processor 250 maycontrol the speaker 230 to output various sound signals (e.g., a voicesignal, a sound effect signal). The processor 250 may control the memory240 to execute a program and retrieve or store necessary information.

According to an embodiment, the processor 250 may execute theintelligent agent 245, the execution manager module 247, and theintelligent service module 249 stored in the memory 240. Therefore, theprocessor 250 may implement the functions of the intelligent agent 245,the execution manager module 247, and the intelligent service module249.

According to an embodiment, the processor 250 may execute theintelligent agent 245 to generate an instruction for operating an app,based on a detection signal received through the mounting of anaccessory. In an embodiment, the processor 250 may execute theintelligent agent 245 to generate an instruction for operating an app,based on a voice signal received through a user input. In an embodiment,the processor 250 may execute the execution manager module 247 toexecute the apps 241 and 243 stored in the memory 240 in accordance withthe generated instruction. In an embodiment, the processor 250 mayexecute the intelligent service module 249 to manage information of theaccessory and process a function for a user input (e.g., the mounting ofthe accessory) by using the information of the accessory. In anembodiment, the processor 250 may execute the intelligent service module249 to manage user information and process a function for a user input(e.g., a user speech) by using the user information.

The processor 250 may execute the intelligent agent 245 to transmit, tothe intelligent server 310, information about a user input (e.g., a userspeech, the mounting of an accessory) received through the input module210 such that the intelligent server 310 can process the user input.

According to an embodiment, the processor 250 may execute theintelligent agent 245 to preprocess the user input before transmittingthe user input to the intelligent server 310. In an embodiment, forpreprocessing of the user input, the intelligent agent 245 may includean adaptive echo canceller (AEC) module, a noise suppression (NS)module, an end-point detection (EPD) module, or an automatic gaincontrol (AGC) module. The AEC module is capable of removing echoes fromthe user input. The NS module is capable of suppressing background noisecontained in the user input. The EPD module is capable of detecting theend point of a user voice contained in the user input and therebyfinding a user voice existing region. The AGC module is capable ofrecognizing the user input and adjusting the volume of the user input tobe suitable for processing. In one embodiment, the processor 250 mayexecute all of the preprocessing modules to realize excellentperformance. In another embodiment, the processor 250 may execute onlysome of the preprocessing modules to reduce power consumption.

According to an embodiment, the intelligent agent 245 may execute awakeup recognition module stored in the memory 240 to recognize a user'scall. Therefore, the processor 250 may recognize a user's wakeup commandthrough the wakeup recognition module and, when the wakeup command isreceived, execute the intelligent agent 245 for receiving a user input.The wakeup recognition module may be implemented as a low-powerprocessor (e.g., a processor included in an audio codec, or an accessorymounting detection sensor). In an embodiment, the processor 250 mayexecute the intelligent agent 245 when receiving a user input through ahardware key. When the intelligent agent 245 is executed, an intelligentapp (e.g., a voice recognition app) linked to the intelligent agent 245may be executed.

According to an embodiment, the intelligent agent 245 may include aspeech recognition module. Through the voice recognition module, theprocessor 250 may recognize a user input for executing an action in anapp. For example, the processor 250 may recognize, through the voicerecognition module, a user input for executing an action such as awakeup command in the apps 241 and 243 (e.g., a user speech such as“click” for executing a shooting action in a camera app). Through thevoice recognition module, the processor 250 may rapidly recognize andprocess a user command and thereby assist the intelligent server 310. Inan embodiment, the voice recognition module of the intelligent agent 245may be implemented in an app processor. In an embodiment, theintelligent agent 245 may include a context awareness module forexecuting a function according to the accessory mounting. Through thecontext awareness module, the processor 250 may recognize the mountingof an accessory. For example, through the context awareness module, theprocessor 250 may recognize context regarding an accessory such as thetype of a mounted accessory, the number of mounted accessories, theposition of a mounted accessory, the time or time zone of mounting anaccessory, or a place where the mounting of an accessory is made.Through the context awareness module, the processor 250 may rapidlyrecognize and process the accessory being mounted on the electronicdevice 101 and thereby assist the intelligent server 310.

According to an embodiment, a recognition module of the intelligentagent 245 (e.g., the voice recognition module, the context awarenessmodule, the wakeup module) is capable of recognizing a user input byusing a suitable algorithm for recognizing voice and/or context. Thisalgorithm may be, for example, at least one of a hidden Markov model(HMM) algorithm, an artificial neural network (ANN) algorithm, or adynamic time warping (DTW) algorithm.

According to an embodiment, the processor 250 may execute theintelligent agent 245 to convert a user's voice input into text data.For example, the processor 250 may transmit a user voice to theintelligent server 310 through the intelligent agent 245 and receivetext data corresponding to the user voice from the intelligent server310. Then, the processor 250 may display the received text data on theexpression device 220. In an embodiment, the processor 250 may executethe intelligent agent 245 to convert an input resulting from themounting of an accessory into data. For example, the processor 250 maytransmit information of the accessory to the intelligent server 310through the intelligent agent 245 and receive data (e.g., emotionexpression data or facial expression data) corresponding to theaccessory information from the intelligent server 310. Then, theprocessor 250 may display the received data on the expression device220.

According to an embodiment, the processor 250 may execute theintelligent agent 245 to receive a path rule from the intelligent server310. In an embodiment, the processor 250 may deliver the path rule tothe execution manager module 247 through the intelligent agent 245.

According to an embodiment, the processor 250 may deliver an executionresult log generated in accordance with the path rule, received from theintelligent server 310, to the intelligence service module 249. Theexecution result log may be accumulated and managed in user's preferenceinformation of the persona module 249 b.

According to an embodiment, the processor 250 may execute the executionmanager module 247 to receive the path rule from the intelligent agent245 and execute the apps 241 and 243 such that the apps 241 and 243execute the actions 241 b and 243 b contained in the path rule. Forexample, the processor 250 may transmit command information (e.g., pathrule information) for executing the actions 241 b and 243 b to the apps241 and 243 through the execution manager module 247 and receivecompletion information of the actions 241 b and 243 b from the apps 241and 243.

According to an embodiment, the processor 250 may execute the executionmanager module 247 to deliver the command information (e.g., path ruleinformation) for executing the actions 241 b and 243 b of the apps 241and 243 between the intelligent agent 245 and the apps 241 and 243. Theprocessor 250 may bind the apps 241 and 243 to be executed in accordancewith the path rule through the execution manager module 247, and deliverthe command information (e.g., path rule information) of the actions 241b and 243 b included in the path rule to the apps 241 and 243. Forexample, the processor 250 may sequentially deliver the actions 241 band 243 b included in the path rule to the apps 241 and 243 through theexecution manager module 247 and thereby sequentially execute theactions 241 b and 243 b of the apps 241 and 243 in accordance with thepath rule.

According to an embodiment, the processor 250 may execute the executionmanager module 247 to manage execution statuses of the actions 241 b and243 b of the apps 241 and 243. For example, the processor 250 mayreceive information about the execution statuses of the actions 241 band 243 b from the apps 241 and 243 through the execution manager module247.

For example, when the execution status of the actions 241 b and 243 b isa stopped status (i.e., partial landing, e.g., when a parameter requiredfor the actions 241 b and 243 b is not entered), the processor 250 maydeliver information about the stopped status to the intelligent agent245 through the execution manager module 247. The processor 250 mayrequest an input of necessary information (e.g., parameter information)from the user by using information received through the intelligentagent 245.

In another example, when the execution status of the actions 241 b and243 b is an operating status, the processor 250 may receive a speechfrom the user or detect the mounting of an accessory through theintelligent agent 245. The processor 250 may deliver, to the intelligentagent 245, information about the apps 241 and 243 (or actions) beingexecuted through the execution manager module 247 and information aboutthe execution status of the apps 241 and 243 (or actions). The processor250 may transmit information about a user's speech or the mounting of anaccessory to the intelligent server 310 through the intelligent agent245. The processor 250 may receive parameter information about a user'sspeech or the mounting of an accessory from the intelligent server 310through the intelligent agent 245. The processor 250 may deliver thereceived parameter information to the execution manager module 247. Theexecution manager module 247 may change parameters of the actions 241 band 243 b to new parameters by using the received parameter information.

According to an embodiment, the processor 250 may execute the executionmanager module 247 to deliver the parameter information included in thepath rule to the apps 241 and 243. When the apps 241 and 243 aresequentially executed in accordance with the path rule, the executionmanager module 247 may deliver the parameter information included in thepath rule from one app to another app.

According to an embodiment, the processor 250 may execute the executionmanager module 247 to receive a plurality of path rules. The processor250 may select the plurality of path rules, based on a user's speechthrough the execution manager module 247. For example, when the user'sspeech specifies one app 241 for executing some actions 241 a and failsto specify the other app 243 for executing other actions 243 b, theprocessor 250 may receive, through the execution manager module 247,different path rules for the specified app 241 (e.g., a gallery app) toexecute the actions 241 a and for non-specified app 243 (e.g., a messageapp, a telegram app) to execute the actions 243 b. In an embodiment, theprocessor 250 may select the plurality of path rules, based on themounting of an accessory through the execution manager module 247. Forexample, when the accessory mounting specifies one app 241 for executingsome actions 241 a and fails to specify the other app 243 for executingother actions 243 b, the processor 250 may receive, through theexecution manager module 247, different path rules for the specified app241 (e.g., a facial expression processing app) to execute the actions241 a and for non-specified app 243 (e.g., a motion processing app, asound processing app) to execute the actions 243 b. The processor 250may execute, for example, the corresponding actions 241 b and 243 b(e.g., successive actions 241 b and 243 b) of the plurality of pathrules through the execution manager module 247. Thereafter, theprocessor 250 may display, on the expression device 220, a status screenfor selecting different apps 241 and 243 included in the plurality ofpath rules.

According to an embodiment, the intelligent service module 249 mayinclude a context module 249 a, a persona module 249 b, and/or aproposal module 249 c.

The processor 250 may execute the context module 249 a to collectcurrent states of the apps 241 and 243 from the apps 241 and 243. Forexample, the processor 250 may execute the context module 249 a toreceive context information indicating the current states of the apps241 and 243 and collect the current states of the apps 241 and 243through the received context information. The processor 250 may executethe context module 249 a to collect a current state of an accessory(e.g., the number of mounted accessories, an accessory mountingposition) and also collect a current environmental state (e.g., a time,a time zone, a place, a location). For example, the processor 250 mayexecute the context module 249 a to receive context informationindicating the current state of the accessory and collect the currentstate of the accessory through the received context information. Inaddition, the processor 250 may execute the context module 249 a toreceive context information indicating the current environmental stateand collect the current environmental state through the received contextinformation.

The processor 250 may execute the persona module 249 b to managepersonal information (e.g., including accessory information) of a userwho uses the electronic device 101. For example, the processor 250 mayexecute the persona module 249 b to collect usage information andexecution results of the electronic device 101 and thereby manage user'spersonal information.

The processor 250 may execute the proposal module 249 c to predict auser's intention and thereby recommend a command to the user. Forexample, the processor 250 may execute the proposal module 249 c torecommend a command to the user according to a user's current state(e.g., a time, a place, a situation, an app, accessory characteristics).

FIG. 4 is a diagram illustrating a process of collecting a current stateat an intelligent service module according to an embodiment of thedisclosure.

Referring to FIG. 4 , when a context request is received from theintelligent agent 245 (as indicated by {circle around (1)}), theprocessor 250 may request context information indicating the currentstates of the apps 241 and 243 through the context module 249 a (asindicated by {circle around (2)}). Then, the processor 250 may receivethe context information from the apps 241 and 243 through the contextmodule 249 a (as indicated by {circle around (3)}) and transmit thereceived context information to the intelligent agent 245 (as indicatedby {circle around (4)}).

According to an embodiment, the processor 250 may receive a plurality ofkinds of context information from the apps 241 and 243 through thecontext module 249 a. The context information may be, for example,information about the most recently executed app (or the most recentlymounted accessory). In another example, the context information may beinformation about the current state in the apps 241 and 243 (e.g.,information about a photo when viewing the photo in a gallery app,information on the type of an accessory, information about the number ofaccessories mounted, information about a mounting position of anaccessory).

According to an embodiment, the processor 250 may receive contextinformation indicating the current state of the electronic device 101from a device platform as well as the apps 241 and 243 through thecontext module 249 a. This type context information may include generalcontext information, user context information, and/or device contextinformation.

The general context information may include general information of theelectronic device 101. The general context information may be obtainedthrough an internal algorithm by receiving data through a sensor hub ofa device platform. For example, the general context information mayinclude information about the current space and time, i.e., about acurrent time and a current location of the electronic device 101.Information about the current time may be obtained through a time in theelectronic device 101, and information about the current location may beobtained through a global positioning system (GPS). In another example,the general context information may include information about a physicalmotion including, for example, walking, running, driving, and the like.The physical motion information may be obtained through a motion sensor.Driving-related information may also be obtained through Bluetoothconnection in a vehicle. In still another example, the general contextinformation may include information about a user activity including, forexample, content viewing, going to school, commute to work, shopping,travel, and the like. The user activity information may be obtainedusing information on content or place registered by the user or the appin a database.

The user context information may include information about the user. Forexample, the user context information may include information about anemotional state of the user. The emotional state information mayinclude, for example, information about happiness, sadness, anger, andthe like. In another example, the user context information may includeinformation about a current state of the user. The current stateinformation may include, for example, information about interest,intention, and the like (e.g., play).

The device context information may include device information about thestate of the electronic device 101. For example, the device contextinformation may include information about a path rule executed by theexecution manager module 247. In another example, the device contextinformation may include information about the battery such as a chargingor discharging state of the battery. In still another example, thedevice context information may include information about a connecteddevice (e.g., an accessory) and a connected network. The connecteddevice information may be obtained via a communication interface towhich the accessory is connected.

Now, terms used in various embodiments of the disclosure will bedescribed.

In various embodiments, the characteristics of an accessory areattributes of the accessory (or assigned to the accessory) and mayinclude, for example, animal, dinosaur, plant, insect, occupation (orrole playing occupation, e.g., police, doctor, scientist, firefighter,sports athlete), character (e.g., cartoon character, angel, demon),linked device function, and the like. In an embodiment, thecharacteristics of an accessory may be identified, based on the type,number, and/or position of the accessory mounted on the electronicdevice 101 (e.g., a robot, hereinafter referred to as a robot) and maybe classified according to stages (e.g., first stage, second stage, orthird stage). In an embodiment, such stages for classifying thecharacteristics of the accessory may be determined, for example, basedon the number and/or positions of mounted accessories. In an embodiment,when the type of a mounted accessory is determined, the robot maydetermine a modeling rule (e.g., a face element (e.g., a bear face, arabbit face, a cartoon character face, a dinosaur face) associated witha specific character assigned to the accessory) corresponding to theaccessory type and provide a change in facial expression stage by stage,based on the modeling rule depending on the number of mountedaccessories.

According to an embodiment, the type of an accessory is one factor fordetermining the characteristics of the accessory and may include, forexample, a character type (e.g., animal, dinosaur, plant, insect,occupation, cartoon character, angel, demon, etc.), an event type (or arole type, e.g., occupation), and a function-connected type (or anexternal device type, e.g., projector, tablet PC, speaker, Internet ofthing (IoT) device, lighting device, camera, etc.).

According to an embodiment, the number of mounted accessories mayindicate the number of accessories mounted on each part of the robot.For example, one or more accessories may be mounted on a head part (A)and a body part (B). Depending on user's intention, accessories havingthe same characteristic or a combination of accessories having differentcharacteristics (e.g., the first characteristic and the secondcharacteristic) may be mounted on the robot. For example, the user maymount ears, arms, and a tail having the first characteristic (e.g., arabbit) on the head part (A) and the body part (B) of the robot. Inanother example, the user may mount ears having the first characteristic(e.g., a rabbit) on the head part (A) of the robot, mount a tail havingsecond characteristic (e.g., a cartoon character) on a first portion(e.g., a tail mountable portion) of the body part (B), and mount armshaving the first, second, or third characteristic on a second portion(e.g., an arm mountable portion) of the body part (B).

According to an embodiment, the mounting position of an accessory mayindicate, for example, portions of each of the head part (A) and thebody part (B) of the robot. In case of the head part (A), the mountingposition may indicate a first portion where an ear-shaped accessory ismounted, a second portion where a head-wearable accessory (e.g., hat,helmet, hood, headband, head lantern) is mounted, or a third portionwhere a face-wearable accessory (e.g. glasses, sunglasses, goggles,mask) is mounted. The first, second, and third portions of the head part(A) may be partly overlapped with or different from each other. In caseof the body part (B), the mounting position may indicate a first portionwhere an arm-shaped accessory is mounted, a second portion where atail-shaped accessory is mounted, a third portion where a neck-wearableaccessory (e.g., tie, scarf, necklace, stethoscope) is mounted, a fourthportion where a back-wearable accessory (e.g., cloak, backpack, tabulaof dinosaur) is mounted, and a fifth portion where an external device ismounted. The first, second, third, fourth, and fifth portions of thebody part (B) may be partly overlapped with or different from eachother.

In various embodiments, the characteristics of an accessory may bedefined, for example, in the form of a lookup table as exemplarily shownin Table 1 below. The lookup table may be stored in at least one of arobot, an accessory, or a server and used in identifying thecharacteristics of an accessory mounted on the robot.

TABLE 1 Accessory Characteristics Type Mounting (Category) MountingNumber Position Classification Unique ID 1^(st) stage: single PriorityCharacteristic 2^(nd) stage: from information two to N − 1 3^(rd) stage:N

As shown in Table 1, the characteristics of an accessory may bedetermined based on, for example, the accessory type, the mountingnumber, and/or the mounting position.

In an embodiment, the type of an accessory may be identified through aunique identifier (ID) assigned to each accessory. For example, aprimary class of unique identifier may be defined as a character type,an event type, or a function-connected type, and a secondary class ofunique identifier may be further defined according to uniquecharacteristic (feature) information of each accessory.

In an embodiment, the mounting number of an accessory may be a factorfor determining the details (or levels) of the accessorycharacteristics. For example, even if a single characteristic isidentified, a feedback level may be differentially provided according tothe details of the identified characteristic depending on the mountingnumber of the accessory.

In an embodiment, the mounting position of an accessory may be a factorfor determining the priority for applying the accessory characteristics.For example, when applying a facial expression based on the accessorycharacteristics, the characteristics of an accessory mounted on the headpart (A) of the robot may be applied with the first priority.

According to an embodiment, the robot may identify the characteristicsof an accessory, based on the lookup table as shown in Table 1. Forexample, the robot may identify characteristic information correspondingto the type, mounting number, and mounting position of the accessoryfrom the lookup table and, based on the identified characteristicinformation, determine the characteristics of the accessory.

FIG. 5 is a diagram illustrating an example of an electronic deviceaccording to an embodiment of the disclosure.

As shown in FIG. 5 , an electronic device (e.g., the electronic device101 in FIG. 1, 2 , or 3) according to various embodiments may beimplemented as a robot (or a robot device). Hereinafter, the electronicdevice 101 will be referred to as a robot 500.

Referring to FIG. 5 , the robot 500 may include a head part (A), a bodypart (B), and a movable part (C).

According to various embodiments, the head part (A) generally has aspherical shape like a head shape (alternatively a square shape, an ovalshape, a diamond shape, or any other shape), and the front surface ofthe head part (A) may include an expression device 510 mounted through afirst housing 501. In an embodiment, the head part (A) may beimplemented to have, at a portion 10 of the first housing 501, amounting member (e.g., magnet, rubber magnet, iron) for allowing anaccessory 600 to be magnetically mounted. In an embodiment, the headpart (A) may include therein at least one circuit for detecting theaccessory and/or communicating with the accessory. This circuit mayinclude sensor circuitry for detecting whether the accessory is mountedor detached, sensor circuitry for detecting the mounting position of theaccessory, and/or communication circuitry for communicating with themounted accessory.

According to an embodiment, as seen from an enlarged view indicated byreference numeral 20 in FIG. 5 , the accessory 600 may include an outputdevice 610, a communication module 620, sensor modules 630 and 640, anda control circuit 650. In addition, the accessory 600 may furtherinclude a housing, and the above-listed components of the accessory 600may be disposed in or on the housing.

According to an embodiment, the output device 610 of the accessory 600may visually provide information to the outside (e.g., the user, theair) of the accessory 600. For example, the output device 610 mayinclude a light emitting device (e.g., light emitting diode (LED), laserdiode (LD)) and/or a hologram device. In an embodiment, the outputdevice 610 of the accessory 600 may tactilely provide information to theoutside of the accessory 600. For example, the output device 610 mayinclude a haptic device.

According to an embodiment, the communication module 620 of theaccessory 600 may operate, for example, as a contactless communicationmodule for contactless communication of the accessory 600. For example,the contactless communication may be implemented with at least onecontactless proximity communication technique such as near fieldcommunication (NFC), radio frequency identification (RFID)communication, or magnetic secure transmission (MST) communication. Inan embodiment, the accessory 600 and the robot 500 may communicate witheach other through any other wireless communication (e.g., Wi-Fi,Bluetooth communication) in addition to the contactless communication,and thus the communication module 620 may further include a suitablecommunication module for supporting such wireless communication. In anembodiment, the communication module 620 of the accessory 600 may beimplemented with an NFC tag. For example, the accessory 600 may inputand store information about the accessory (e.g., an accessory uniqueidentifier) in the NFC tag, and the robot 500 may include acommunication circuit 520 (e.g., NFC module) capable of readinginformation stored in the NFC tag of the accessory 600.

According to an embodiment, the sensor modules 630 and 640 of theaccessory 600 may include, for example, a module that generates a signalfor detecting a mounting position of the accessory 600 in the robot 500.For example, the sensor modules 630 and 640 may include a magneticobject (e.g., a magnet or a rubber magnet) that affects current. In anembodiment, the sensor modules 630 and 640 may operate as mountingmembers. For example, when the sensor modules 630 and 640 areimplemented as magnets (or rubber magnets), they may be magneticallyattached to the mounting member (e.g., iron) provided on the housing ofthe robot 500.

In various embodiments, the expression device 510 (e.g., the displaydevice 160 in FIG. 1 or the expression device 220 in FIG. 3 ) of thehead part (A) may be exposed through a portion (e.g., front surface) ofthe first housing 501. In an embodiment, the expression device 510 mayvisually provide a facial expression based on a face-related object(e.g., graphical elements such as eyes, nose, mouth, or teeth) orvisually provide an object (e.g., image, video, or text) related to afunction (or service) of the robot 500. In various embodiments, when noaccessory is mounted on the robot 500, the expression device 510 mayimplement and display a default face (or facial expression)corresponding to default characteristics set in the robot 500. Then, inresponse to an interaction with the user, the default face or facialexpression may be changed and displayed. In various embodiments, inorder to express the characteristics of an accessory mounted on therobot 500, the expression device 510 may characterize at least onerelated object (e.g., a face shape) corresponding to thecharacteristics. For example, the expression device 510 may differentlydisplay the shape, number, and/or arrangement of at least oneface-related object (e.g., eyes, nose, mouth, teeth) so as to expressthe characteristics of the mounted accessory. In an embodiment, withrespect to the characterized object (e.g., a face) corresponding to thecharacteristics of the accessory, the expression device 510 may visuallyprovide a change in the facial expression in response to an interactionwith the user. For example, the expression device 510 may visuallyprovide various changes in the facial expression, such as a smilingface, a crying face, an angry face, a sad face, an expressionless face,a surprised face, a frowning face, or a thinking face, based on thecharacterized object having the corresponding characteristics.

According to an embodiment, the expression device 510 may fully expressa face of the head part (A) through the display, or express only someobjects of a face (e.g., eyes, pupils, eyebrows, nose, mouth) throughthe display and express the other objects (e.g., skin) through aspecific material (e.g., silicone or rubber). In an embodiment, theexpression device 510 may express only a portion corresponding to pupilsthrough the display and express other portions such as eyebrows, nose,and mouth through a motion device or an actuator for expressing anactual motion. In an embodiment, the expression device 510 may basicallyinclude the housing of a specific material (e.g., silicone or rubber)and further include both the display for visually offering a facialexpression and the motion device (or actuator) for actually offering amotion-based facial expression. In an embodiment, a face of the headpart (A) may be formed of a specific material (e.g., silicone orrubber), and some holes may be formed at points corresponding toface-related objects (e.g., eyes, eyebrows, nose, mouth) so as toaccommodate respective expression devices 510. For example, left andright eyes of the robot may be implemented through respective individualexpression devices or a single expression device.

According to various embodiments, the body part (B) may have a generallytorso shape. The body part (B) may be implemented to have, at a portionof a second housing 503, a mounting member (e.g., magnet, iron) forallowing an accessory to be magnetically mounted. In an embodiment, thebody part (B) may include therein at least one circuit for detecting theaccessory and/or communicating with the accessory. This circuit mayinclude sensor circuitry for detecting whether the accessory is mountedor detached, sensor circuitry for detecting the mounting position of theaccessory, and/or communication circuitry for communicating with themounted accessory.

According to an embodiment, as seen from the enlarged view 20 in FIG. 5, the robot 500 may detect the magnetic field formed by the sensormodules 630 and 640 of the accessory 600 through first sensor circuits530 and 540 (e.g., a hall sensor, or a Hall Effect sensor). In anembodiment, the first sensor circuits 530 and 540 may include varioussensors for contactless position detection using the Hall Effect. In anembodiment, a processor 550 of the robot 500 may identify the firstsensor circuits 530 and 540 detecting the magnetic field and therebyidentify the mounting position of the accessory 600.

According to various embodiments, the robot 500 may acquire informationabout the accessory 600 (e.g., an accessory unique identifier or ID) bycommunicating with the communication module 620 (e.g., an NFC tag) ofthe accessory 600 via the communication circuit 520, and the processor550 may identify the type of the accessory 600 mounted on the robot 500based on the acquired information about the accessory 600. In variousembodiments, the first sensor circuits 530 and 540 and the communicationcircuit 520 of the robot 500 may be installed at each of respectiveportions of the head part (A) and the body part (B) where accessoriesare mounted. The mounting positions where the accessories can be mountedon the robot 500 will be described below with reference to FIG. 6 .

According to an embodiment, the body part (B) may include a mountingmember (e.g. a magnet or iron) at each accessory-mountable portion onfront, rear, left and/or right surfaces of the second housing 503. In anembodiment, the body part (B) may include various components inside thesecond housing 503, such as the processor 550, a speaker 560, a secondsensor circuit 570 (e.g., a distance sensor), and/or a light emittingdevice 580 (e.g., LED, LD, or hologram device). The body part (B) mayfurther include various additional components (e.g., some components ofthe electronic device 101 in FIG. 1 or 3 ). In an embodiment, the bodypart (B) may have a microphone (not shown) and the speaker 560 disposedin the second housing 503, and holes corresponding to the microphone andthe speaker 560 may be formed on the second housing 503 (e.g., the frontsurface of the body part (B)) to facilitate a sound input into themicrophone and a sound output from the speaker 560.

According to an embodiment, the body part (B) may be divided, forexample, into a first body part (B1) corresponding to the neck, a secondbody part (B2) corresponding to the chest and/or the abdomen, and athird body part (B3) corresponding to the lower body. The second bodypart (B2) may be interposed between and supported by the first body part(B1) and the third body part (B3). In an embodiment, the first body part(B1) and the third body part (B3) may be implemented in a hard form, andthe second body part (B2) may be implemented in a soft form. Forexample, the first body part (B1) and the third body part (B3) may beformed of a hard material (e.g., aluminum, aluminum alloy, iron, or thelike) to allow fixed mounting of accessories. In addition, the secondbody part (B2) may be formed of a soft, elastic, and shock-absorbablematerial (e.g., silicone, thermoplastic poly urethane (TPU),thermoplastic elastomer (TPE), or the like).

According to an embodiment, the body part (B) may include at least onesecond sensor circuit 570 (e.g., a distance sensor such as an infraredsensor or an ultrasonic sensor) inside the second housing 503. In anembodiment, for sensing in N directions (where N is a natural number) ofthe second body part (B2), the second sensor circuit 570 may beimplemented with N sensors. For example, the second sensor circuit 570may be composed of four distance sensors for sensing in four directionson the front, rear, left, and right surfaces. In another example, thesecond sensor circuit 570 may be composed of two distance sensors forsensing in two directions on the left and right surfaces or front andrear surfaces. In an embodiment, the second sensor circuit 570 mayinclude one or more sensors capable of detecting (or receiving) a userinput (e.g., a touch, a hug, a stroke) as a gesture signal through thebody part (B). In an embodiment, the second sensor circuit 570 of thesecond body part (B2) may measure a distance between at least two pointsby including a triangulation-based sensor (e.g., using infrared light ornatural light) or an ultrasonic sensor. For example, this sensor maytransmit a signal (e.g., an infrared ray or an ultrasonic wave) andmeasure a return time of the signal reflected by a measurement target.Then, the processor 550 may receive the measured distance from thesecond sensor circuit 570, detect a user input for the second body part(B2), and process a relevant feedback (e.g., a motion of the second bodypart (B2), an expression through the expression device 510, a soundoutput through the speaker 560).

According to various embodiments, the processor 550 may detect themounting of the accessory 600, identify a type of the mounted accessory,a mounting position of the accessory, and/or the number of mountedaccessories, identify the characteristics of the accessory based on theidentified type, position, and/or number, and provide various feedbacksabout the accessory 600 to the user according to the characteristics ofthe accessory. In an embodiment, the processor 550 may provide suchfeedbacks stage by stage in response to the mounting of the accessory600 and in response to an interaction between the user and the robot500. The feedbacks may include, for example, expressions based onchanges in display theme, facial expression, sound, motion (or behavior,appearance), haptic, lighting, and/or color of the robot 500. Variousfeedbacks about the accessory 600 will be described below with referenceto figures.

According to an embodiment, the first body part (B1) and the third bodypart (B3) may be implemented in a hard form such that the accessory 600can be mounted. In addition, the second body part (B2) may beimplemented in a soft form so as to provide feedbacks (e.g., twisting,bending, or shaking the body) related to interactions with the user. Inan embodiment, the robot 500 may include various components (not shown,e.g., a motor, a driving circuit such as a motion controller, a motionsensor, a driver, and/or the like) necessary for driving the robot 500inside the first housing 501 and/or the second housing 503.

According to various embodiments, the movable part (C) may be formedwith wheels 502 under the body part (B) and enable the movement (e.g.,driving, walking) of the robot 500. In various embodiments, the movablepart (C) may be designed to improve the movement stability andsteerability of the robot 500.

FIG. 6 is a diagram illustrating mountable positions of an accessory ona robot according to an embodiment of the disclosure.

Referring to FIG. 6 , the robot 500 may allow an accessory to be mountedon the head part (A) and the body part (B). In an embodiment, each ofthe head part (A) and the body part (B) may be divided into regions (orportions) where accessories can be mounted. These regions or portionsare or include mountable positions of accessories.

According to an embodiment, the head part (A) may include a top portion601, a back portion 603, a face (or front) portion 605, and/or a sideportion 607. The side portion 607 of the head part (A) may be dividedinto a head-left portion and a head-right portion.

According to an embodiment, the body part (B) may include a frontportion 611, a back portion 613, a hip portion 615, and/or a sideportion 617. The side portion 617 of the body part (B) may be dividedinto a body-left portion and a body-right portion.

Various embodiments of the disclosure are not limited to the abovedivisions shown in FIG. 6 .

In case of the head part (A), the mountable positions of accessories mayinclude a first portion where an ear-shaped accessory is mounted, asecond portion where a head-wearable accessory (e.g., hat, helmet, hood,headband, head lantern) is mounted, and/or a third portion where aface-wearable accessory (e.g. glasses, sunglasses, goggles, mask) ismounted. The first, second, and third portions of the head part (A) maybe partly overlapped with or different from each other.

In case of the body part (B), the mountable positions of accessories mayinclude a first portion where an arm-shaped accessory is mounted, asecond portion where a tail-shaped accessory is mounted, a third portionwhere a neck-wearable accessory (e.g., tie, scarf, necklace,stethoscope) is mounted, a fourth portion where a back-wearableaccessory (e.g., cloak, backpack, tabula of dinosaur) is mounted, and afifth portion where an external device is mounted. The first, second,third, fourth, and fifth portions of the body part (B) may be partlyoverlapped with or different from each other.

According to an embodiment, the mountable positions of character-typeaccessories may be differentiated according to characters. For example,an accessory may be mounted on at least one corresponding portionsuitable for applying the characteristics of each character, and suchpositions may be different according to characters.

For example, in case of accessories of a rabbit character, left andright ear accessories may be mounted on the top portion 601 of the headpart (A), and a tail accessory may be mounted on the hip portion 615 ofthe body part (B). In case of accessories of a bear character, left andright ear accessories may be mounted on the top portion 601 of the headpart (A), a tail accessory may be mounted on the hip portion 615 of thebody part (B), and an abdomen accessory may be mounted on the frontportion 611 of the body part (B). In case of accessories of a kangaroocharacter, left and right ear accessories may be mounted on the topportion 601 of the head part (A), a tail accessory may be mounted on thehip portion 615 of the body part (B), and a pocket accessory may bemounted on the front portion 611 of the body part (B). In case ofaccessories of a dinosaur character, left and right ear accessories maybe mounted on the top portion 601 of the head part (A), a teethaccessory may be mounted on the front portion 605 of the head part (A),a horn accessory may be mounted on the back portion 603 of the head part(A), a tabula accessory may be mounted on the back portion 613 of thebody part (B), and a tail accessory may be mounted on the hip portion615 of the body part (B).

According to various embodiments, a combination of accessories havingparticular characteristics may be variously configured according to anaccessory design or a user's choice, such as a combination of ears andtail, a combination of ears, arms, and tail, or a combination of ears,horns, arms, abdomen, and tail.

FIG. 7 is a block diagram illustrating a robot and an accessoryaccording to an embodiment of the disclosure.

Referring to FIG. 7 , the robot 500 according to an embodiment mayinclude the processor 550, a reader 701, a sensor circuit 703, acommunication circuit 705, an output module 707, an interaction module709, and a memory 711.

Although FIG. 7 schematically illustrates the robot 500, the robot 500according to various embodiments may include some or all of thecomponents of the electronic device 101 illustrated in FIG. 1 or 5 . Forexample, the robot 500 may be the same type device as or a differenttype device from the electronic device 101 shown in FIG. 1 or 5 .According to an embodiment, the processor 550, the reader 701, thesensor circuit 703, the communication circuit 705, the output module707, the interaction module 709, and the memory 711 of the robot 500 mayperform all or some of operations of corresponding components of theelectronic device 101 shown in FIG. 1 or 5 .

According to an embodiment, the processor 550 may be operativelyconnected to the above components of the robot 500. In an embodiment,the processor 550 may execute software to control any other component(e.g., hardware or software component) of the robot 500 connectedthereto and perform data processing. For example, the processor 550 mayreceive instructions or data from any other component (e.g., the reader701, the sensor circuit 703, or the communication circuit 705), load thereceived instructions or data in the memory 711 for processing, andstore resultant data in the memory 711.

According to various embodiments, the processor 550 may control overalloperations of the robot 500. For example, the processor 550 may detectthe mounting of an accessory 600, identify the characteristics of theaccessory 600 (e.g., an accessory type, an accessory mounting position,the number of mounted accessories) in response to the detection of theaccessory 600, and output a corresponding feedback based on thecharacteristics of the accessory 600. In an embodiment, the processor550 may control the output module 707 to display an image (e.g., a faceor a facial expression) or output various sound signals (e.g., a voicesignal, a sound effect signal). In an embodiment, the processor 550 maycontrol the memory 711 to execute a program and retrieve or storenecessary information.

According to various embodiments, the processor 550 may detect themounting of at least one accessory 600, identify accessorycharacteristics associated with the at least one accessory 600,determine properties of the robot 500 associated with the at least oneaccessory 600 based on the accessory characteristics, change theproperties of the robot 500 based on the determined properties, andoutput at least one of a visual element, an auditory element, or atactile element associated with the at least one accessory 600 based onthe changed properties.

According to various embodiments, the processor 550 may identify thecharacteristics of the accessory 600 by using a learning model trainedthrough an artificial intelligence (AI) algorithm. In an embodiment, theAI algorithm may include, for example, a machine learning algorithm, aneural network algorithm, a genetic algorithm, a deep learningalgorithm, or a classification algorithm. In an embodiment, theprocessor 550 may receive a result of identifying the characteristics ofthe mounted accessory 600 from a server 700. For example, the processor550 may transmit characteristic information (e.g., an accessory ID, thenumber of mounted accessories, and a mounting position) about themounted accessory 600 to the server 700 and then receive, from theserver 700, information about the accessory characteristics identifiedbased on the transmitted information.

According to an embodiment, the reader 701 may operate as a contactlesscommunication module for contactless communication of the robot 500. Forexample, the contactless communication may be implemented with at leastone contactless proximity communication technique such as near fieldcommunication (NFC), radio frequency identification (RFID)communication, or magnetic secure transmission (MST) communication. Inan embodiment, the reader 701 may be implemented as, for example, an NFCreader (or NFC module). For example, the reader 701 may read information(e.g., a unique identifier or ID) about the accessory 600 stored in atag 721 (e.g., NFC tag) of the accessory 600.

According to an embodiment, the sensor circuit 703 may include at leastone of circuitry for detecting whether the accessory 600 is mounted ordetached, circuitry for detecting the mounting position of the accessory600, or circuitry for detecting a user input. In an embodiment, thesensor circuit 703 may include a first sensor circuit (e.g., a hallsensor, or a Hall Effect sensor) for detecting the mounting of theaccessory 600 and/or the mounting position, and a second sensor circuit(e.g., a distance sensor such as an infrared sensor or an ultrasonicsensor) for detecting (or receiving) a user input (e.g., touch, hug,stroke) through the robot 500 (e.g., the head part (A) and the body part(B)).

According to an embodiment, for sensing in N directions (where N is anatural number) of the robot 500, the second sensor circuit may beimplemented with N sensors. For example, the second sensor circuit maybe composed of four distance sensors for sensing in four directions onthe front, rear, left, and right surfaces of the robot 500. In anotherexample, the second sensor circuit may be composed of two distancesensors for sensing in two directions on the left and right surfaces orfront and rear surfaces.

According to an embodiment, the communication circuit 705 may establisha wireless communication channel between the robot 500 and the accessory600 or between the robot 500 and the server 700 (e.g., a cloud server, amanufacturer server) and perform wireless communication through theestablished communication channel. In an embodiment, the communicationcircuit 705 may communicate with a communication module 723 of theaccessory 600 to acquire (or receive) information about the accessory600 (e.g., a unique identifier or ID of the accessory).

According to an embodiment, the output module 707 may visually provide(or display or output) information to the outside (e.g., a user) of therobot 500. In an embodiment, the output module 707 may include one ormore modules capable of outputting user experience (UX) features of therobot 500 (e.g., display (or display theme), face, motion, lighting,sound, voice, haptic, and/or color) according to the characteristics ofthe accessory 600. For example, the output module 707 may include adisplay device, a haptic device, a motion device, a sound output device,a lighting device, and/or a color material or color control device.

According to an embodiment, the interaction module 709 may perform aninteraction between the robot 500 and the accessory 600 or between therobot 500 and the user. In an embodiment, the interaction module 709 maybe included in the processor 550. In an embodiment, the interactionmodule 709 may provide a feedback as a dynamic (or immediate) responseto the interaction with the user to improve the emotional usability ofthe robot 500.

According to an embodiment, the memory 711 may store various data usedby at least one component of the robot 500 (e.g., the processor 550).Such data may include, for example, software (e.g., the program 140 inFIG. 1 ) and input/output data for related instructions. The memory 711may include a volatile memory and/or a nonvolatile memory. In anembodiment, the memory 711 may store a module for performing processesor functions according to various embodiments, and the module stored inthe memory 711 may be executed by the processor 550. In an embodiment,the memory 711 may store a lookup table relating to accessorycharacteristics as shown in Table 1 above.

The accessory 600 according to an embodiment may include the controlcircuit 650, the tag 721, the communication module 723, a sensor module725, an output device 727, an interaction module 729, and a memory 731.

According to an embodiment, the control circuit 650 may be operativelyconnected to the above components of the accessory 600. In anembodiment, the control circuit 650 may execute software to control anyother component (e.g., hardware or software component) of the accessory600 connected thereto and perform data processing. According to variousembodiments, the control circuit 650 may control the overall operationof the accessory 600. For example, the control circuit 650 may receive acontrol command from the robot 500 and output a feedback correspondingto the control command. In an embodiment, the control circuit 650 maycontrol the output device 727 to output various sound signals (e.g., avoice signal, a sound effect signal), haptic signals, or lighting. In anembodiment, the control circuit 650 may control the memory 711 toexecute a program and retrieve or store necessary information.

According to an embodiment, the tag 721 may operate as a contactlesscommunication module for contactless communication of the accessory 600.For example, the contactless communication may be implemented with atleast one contactless proximity communication technique such as NFC,RFID communication, or MST communication. In an embodiment, the tag 721may be implemented as an NFC tag. For example, the accessory 600 mayenter and store information (e.g., a unique identifier or ID) about theaccessory in the NFC tag.

According to an embodiment, the communication module 723 may include awireless communication module for supporting wireless communication withthe robot 500. In an embodiment, the communication module 723 mayestablish a wireless communication channel between the accessory 600 andthe robot 500 and perform wireless communication through the establishedcommunication channel. In an embodiment, the communication module 723may communicate with the communication circuit 705 of the robot 500 toacquire (or receive) control information (or control command or feedbackinformation) about the control of the accessory 600.

According to an embodiment, the sensor module 725 may include a modulefor generating a signal for detecting the mounting position of theaccessory 600 in the robot 500. For example, the sensor module 725 mayinclude a magnetic object (e.g., a magnet or a rubber magnet) thataffects current. In an embodiment, the sensor module 725 may operate asa mounting member. For example, when the sensor module 725 isimplemented as a magnet (or rubber magnet), it may be magneticallyattached to the mounting member (e.g., iron) provided at anaccessory-mountable position on the housing of the robot 500.

According to an embodiment, the output device 727 may visually and/ortactilely provide information to the outside (e.g., the user) of theaccessory 600. For example, the output device 727 may include a lightemitting device (e.g., LED, LD), a hologram device, or a haptic device.

According to an embodiment, the interaction module 729 may perform aninteraction between the accessory 600 and the robot 500 or between theaccessory 600 and the user. In an embodiment, the interaction module 729may be included in the control circuit 650. In an embodiment, theinteraction module 729 may provide a feedback as a dynamic (orimmediate) response to the interaction with the user to improve theemotional usability of the accessory 600.

According to an embodiment, the memory 731 may store various data usedby at least one component of the accessory 600 (e.g., the controlcircuit 650). Such data may include, for example, software (e.g., theprogram 140 in FIG. 1 ) and input/output data for related instructions.The memory 731 may include a volatile memory and/or a nonvolatilememory. In an embodiment, the memory 731 may store a module forperforming processes or functions according to various embodiments, andthe module stored in the memory 731 may be executed by the controlcircuit 650. In an embodiment, the memory 731 may store a lookup tablerelating to accessory characteristics as shown in Table 1 above.

According to an embodiment, the server 700 may include an externalserver such as a cloud server, a manufacturer server, or a contentproviding server. In an embodiment, the server 700 (e.g., a cloudserver) may store and manage information about various accessories beingmountable on the robot 500. In an embodiment, the server 700 mayinteroperate with the robot 500 to store or manage information receivedfrom the robot 500 and provide the robot 500 with information requestedby the robot 500. In an embodiment, information about accessories mayinclude, for example, characteristics of each accessory and be recordedin a lookup table as shown in Table 1 above.

According to an embodiment, the server 700 may receive characteristicinformation (e.g., an accessory ID, the number of mounted accessories,and a mounting position) about the mounted accessory 600 from the robot500, identify the accessory characteristics based on the receivedcharacteristic information, and provide information about the identifiedaccessory characteristic to the robot 500. In an embodiment, the server700 may provide relevant resources (e.g., information or big data aboutthe accessory) to the robot 500, based on virtualization and distributedprocessing.

In an embodiment, the robot 500 and/or the accessory 600 may not includeat least one of the above-described components (e.g., the memory 711 or731, and/or a certain circuit) or may further include any othernon-described component. In an embodiment, some of the above-describedcomponents may be implemented in one integrated circuit. In anembodiment, the server 700 may not be used, or one or more additionalservers may be used.

According to various embodiments of the disclosure, an electronic devicemay include a housing, at least one sensor circuit disposed in thehousing and configured to detect mounting of at least one accessory, atleast one communication circuit disposed in the housing and configuredto communicate with the at least one accessory, a display devicedisposed in the housing and configured to display visual informationassociated with the at least one accessory, and a processor disposed inthe housing and operatively connected to the sensor circuit, thecommunication circuit, and the display device. In particular, theprocessor may be configured to detect the mounting of the at least oneaccessory, to identify accessory characteristics associated with the atleast one accessory, to determine properties of the electronic deviceassociated with the at least one accessory, based on the accessorycharacteristics, to change the properties of the electronic device,based on the determined properties, and to output at least one of avisual element, an auditory element, or a tactile element associatedwith the at least one accessory, based on the changed properties.

According to various embodiments, the processor may be furtherconfigured to acquire accessory information from the at least oneaccessory through the communication circuit when the mounting of the atleast one accessory is detected through the communication circuit, andto identify the accessory characteristics, based on the accessoryinformation.

According to various embodiments, the processor may be furtherconfigured to identify the accessory characteristics, based on at leastone of a type of accessory, the number of mounted accessories, or amounting position of accessory, or a combination thereof.

According to various embodiments, the processor may be furtherconfigured to identify the number of mounted accessories, and to providea feedback by realizing an intention corresponding to the identifiednumber of mounted accessories.

According to various embodiments, the processor may be furtherconfigured to determine the properties of the electronic device withdifferent degrees of completion depending on the identified number ofmounted accessories.

According to various embodiments, the processor may be furtherconfigured to classify the accessory characteristics stage by stagedepending on the identified number of mounted accessories.

According to various embodiments, the processor may be furtherconfigured to determine a modeling rule corresponding to the type ofaccessory, and to provide a feedback based on the modeling rule stage bystage depending on the identified number of mounted accessories.

According to various embodiments, the processor may be furtherconfigured to provide a feedback associated with the mounting of the atleast one accessory, based on the electronic device or the at least oneaccessory.

According to various embodiments, the processor may be furtherconfigured to determine a modeling rule corresponding to the accessorycharacteristics, and to provide a feedback as an immediate response toan interaction with a user, based on the modeling rule.

According to various embodiments, the processor may be furtherconfigured to identify the accessory characteristic by using a learningmodel trained through an artificial intelligence algorithm.

Now, an operating scenario of the robot 500 which corresponds to theelectronic device 101 will be described. Although it will be describedthat the operations of the robot 500 are controlled by the processor 550shown in FIG. 5 or 7 , the processor 550 may include all or part of theprocessor 120 shown in FIG. 1 or the processor 250 shown in FIG. 2 . Insome embodiments, the processor 550 may be referred to as at least oneprocessor or control circuit that includes processing circuitry, orreferred to as instructions that are stored in a memory and cause, whenexecuted, the processor 550 to operate.

FIG. 8 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure.

Referring to FIG. 8 , at operation 801, the processor 550 of the robot500 may identify characteristics of an accessory. According to anembodiment, when the mounting of the accessory is detected, theprocessor 550 may identify the type of the mounted accessory, the numberof the mounted accessories, and the position of the mounted accessory. Adetailed operation will be described below with reference to FIG. 9 .

According to an embodiment, the processor 550 may identify accessorycharacteristics by using a lookup table as shown in Table 1. In anembodiment, the lookup table may be stored, for example, in the robot500, an accessory (e.g., the accessory 600 in FIG. 7 ), or an externalserver (e.g., the server 700 in FIG. 7 ). When the robot 500 stores alookup table, the processor 550 may identify the characteristics of theaccessory through the lookup table by using information (e.g., a uniqueidentifier or ID) obtained from the accessory. When the server 700stores a lookup table, the processor 550 may provide the server 700 withinformation (e.g., a unique identifier or ID) obtained from theaccessory, and receive information about the characteristics of theaccessory from the server 700. When the accessory stores a lookup table,the processor 550 may receive information about the characteristics ofthe accessory from the accessory.

In various embodiments, the characteristics of an accessory areattributes of the accessory (or assigned to the accessory) and mayinclude, for example, animal, dinosaur, plant, insect, occupation (orrole playing occupation, e.g., police, doctor, scientist, firefighter,sports athlete), character (e.g., cartoon character, angel, demon),linked device function, and the like. In an embodiment, thecharacteristics of an accessory may be identified, based on the type,number, and/or position of the accessory mounted on the robot 500.

According to an embodiment, the type of an accessory is one of factorsfor determining the characteristics of the accessory and may include,for example, a character type (e.g., animal, dinosaur, plant, insect,occupation, cartoon character, angel, demon, etc.), an event type (or arole type, e.g., occupation), and a function-connected type (or anexternal device type, e.g., projector, tablet PC, speaker, Internet ofthing (IoT) device, lighting device, camera, etc.).

According to an embodiment, the number of mounted accessories mayindicate the number of accessories mounted on each part of the robot500. For example, one or more accessories may be mounted on a head part(A) and a body part (B). Depending on user's intention, accessorieshaving the same characteristic or a combination of accessories havingdifferent characteristics (e.g., the first characteristic and the secondcharacteristic) may be mounted on the robot.

According to an embodiment, the mounting position of an accessory mayindicate, for example, portions of each of the head part (A) and thebody part (B) of the robot. In case of the head part (A), the mountingposition may indicate a first portion where an ear-shaped accessory ismounted, a second portion where a head-wearable accessory is mounted, ora third portion where a face-wearable accessory is mounted. The first,second, and third portions of the head part (A) may be partly overlappedwith or different from each other. In case of the body part (B), themounting position may indicate a first portion where an arm-shapedaccessory is mounted, a second portion where a tail-shaped accessory ismounted, a third portion where a neck-wearable accessory is mounted, afourth portion where a back-wearable accessory is mounted, and a fifthportion where an external device is mounted. The first, second, third,fourth, and fifth portions of the body part (B) may be partly overlappedwith or different from each other.

At operation 803, the processor 550 may determine the properties of therobot 500, based on the accessory characteristics. In an embodiment, theprocessor 550 may determine the properties of the robot 500 as thecharacteristics of the accessory, for example, animal, dinosaur, plant,insect, occupation (e.g., police, doctor, scientist, firefighter, sportsathlete), character (e.g., cartoon character, angel, demon), linkeddevice function, and the like.

At operation 805, the processor 550 may provide feedback, based on theproperties of the robot 500. In an embodiment, the processor 550 maycontrol at least one component to output the feedback in accordance withthe determined properties of the robot 500. For example, depending onthe properties of the robot 500, the processor 550 may provide at leastone feedback (e.g., a change in facial expression, a change in motion, achange in sound (or voice), lighting, and/or the like) to the userthrough an output module (e.g., the output module 707 in FIG. 7 ). Thefeedback may be selected according to suitable components of the robot500.

FIG. 9 is a diagram illustrating a method for identifyingcharacteristics of an accessory at a robot according to an embodiment ofthe disclosure.

FIG. 9 illustrates operation 801 where the processor 550 of the robot500 may identify characteristics of an accessory. Referring to FIG. 9 ,when the mounting of an accessory is detected, the processor 550 of therobot 500 may identify the characteristics of the accessory byidentifying the type of the mounted accessory at operation 910,identifying the number of the mounted accessories at operation 920, andidentifying the position of the mounted accessory at operation 930. Invarious embodiments, such operations 910, 920, and 930 may be performedsequentially as shown in FIG. 9 or in another order, or in parallel. Atleast one such operation may be omitted, or any additional operation maybe further performed.

According to various embodiments, based on the accessory type, theaccessory mounting number, and/or the accessory mounting position, theprocessor 550 may apply each corresponding characteristic or provide arelevant feedback. In an embodiment, the processor 550 may identify theaccessory type and provide a feedback by applying characteristics andfunctions corresponding to the identified accessory type. In anembodiment, the processor 550 may identify the accessory mounting numberand provide a feedback by realizing the intention corresponding to theidentified mounting number. In an embodiment, the processor 550 maydetermine the properties of the robot with different degrees ofcompletion depending on the identified number of mounted accessories. Inan embodiment, the processor 550 may identify the accessory mountingposition and provide a feedback corresponding to the identified mountingposition. In an embodiment, the processor 550 may determine whether theaccessory is mounted at a correct position or a wrong position, and thusprovide a relevant feedback.

According to various embodiments, at operation 910, the processor 550may identify the type of the mounted accessory, as follows.

In an embodiment, when the mounting of a certain accessory is detected,the processor 550 may identify the type of the detected (i.e., currentlymounted) accessory. At this time, the processor 550 may identify theaccessory type in consideration of the detected (i.e., currentlymounted) accessory and the previously mounted accessory. For example,the processor 550 may identify the first type of the detected accessoryand determine whether the identified first type is identical to thesecond type of the previously mounted accessory. If the first and secondtypes are identical, the processor 550 may determine that the same typeaccessory is mounted.

If it is determined that another type of accessory is currently mounted,the processor 550 may provide a relevant feedback predefined accordingto setting. For example, the processor 550 may provide the user withvisual, audio, and/or tactile information informing that the detectedaccessory is of a different type. In another example, the processor 550may provide a combined feedback by combining a feature of the first typeof the currently detected accessory with a feature (e.g., UX feature orsound) of the second type of the previously mounted accessory.

According to various embodiments, as shown in Table 2 below, varioustypes of accessories may be classified into some categories (or classes)according to a certain criterion, and relevant feedbacks may bepredefined.

TABLE 2 Category 1^(st) Category 2^(nd) Category 3^(rd) CategoryCriterion Character Type Event Type Function- Connected Type Feedback1^(st) Style 2^(nd) Style 3^(rd) Style Feedback Feedback Feedback

Referring to Table 2, the first category may contain character-typeaccessories that represent user-friendly characters such as, forexample, animal, dinosaur, plant, insect, occupation (or role, e.g.,police, doctor, scientist, firefighter, sports athlete), cartooncharacter, game character, angel, demon, and the like.

The second category may contain event-type accessories (e.g. hat,clothing, cloak, tie, necklace, glasses, stethoscope, etc.) associatedwith a particular event (e.g., Christmas, Halloween, birthday,country-specific costume, specific effect (e.g., color variation,pattern variation, material change), etc.) or accessories forconvenience (e.g., cup holders, storage boxes, etc.).

The third category may contain function-connected accessories forextending the functionality of the robot 500 by combining (orfunctionally connecting) an external device (for providing specificcontent and/or service) such as a projector, a tablet PC, a speaker, aninternet of thing (IoT) device, a lighting device, or a camera.

According to an embodiment, the processor 550 may provide a first stylefeedback in case of accessories of the first category. For example, forthe accessories of the first category, the processor 550 may provide afeedback by changing default properties (e.g., facial expression, sound(e.g., voice), etc.) of the robot 500 to correspond to the accessorycharacteristics.

According to an embodiment, the processor 550 may provide a second stylefeedback in case of accessories of the second category. For example, forthe accessories of the second category, the processor 550 may provide afeedback by adding an event-related expression and sound whilemaintaining the default properties of the robot 500 or the propertieschanged according to the accessories of the first category.

According to an embodiment, the processor 550 may provide a third stylefeedback in case of accessories of the third category. For example, forthe accessories of the third category, the processor 550 may provide afeedback by adding or expanding a function while maintaining the defaultproperties of the robot 500 or the properties changed according to theaccessories of the first category.

According to an embodiment, different-type accessories may be mounted onthe robot 500 in combination. In an embodiment, when different-typeaccessories are mounted in combination, the processor 550 may apply anexpression according to the characteristics of one accessory mounted onthe head part (A), and also apply a motion according to thecharacteristics of another accessory mounted on the body part (B). In anembodiment, when different-type accessories are mounted together in thesame region (e.g., the head part (A) or the body part (B)), theprocessor 550 may preferentially apply the characteristics of theaccessory firstly mounted on that region. In an embodiment, whendifferent-type accessories are mounted together in one region (e.g., thebody part (B)), the processor 550 may apply in combination therespective characteristics of the different-type accessories. In thiscase, the priorities of applying the characteristics of thedifferent-type accessories may be set by the user (e.g., first priority:first category, second priority: third category, and third priority:second category, or first priority: first character (e.g., cartooncharacter), second priority: second character (e.g., bear), and thirdpriority: third character (e.g., a dinosaur)).

According to an embodiment, when the accessory of the second category orthe accessory of the third category is mounted after the characteristicsof the accessory of the first category is applied, the characteristicsof the accessory of the second or third category may be applied while aportion changed by applying the characteristics of the accessory of thefirst category is maintained as it is.

According to an embodiment, when the accessory of the third category ismounted after the accessories of the first and second categories areapplied, the characteristics of the accessory of the third category maybe applied while portions changed by applying the characteristics of theaccessories of the first and second categories are maintained as theyare.

According to various embodiments, at operation 920, the processor 550may identify the number of the mounted accessories, as follows.

In an embodiment, when the mounting of a certain accessory is detected,the processor 550 may identify the number of mounted accessories inconsideration of the detected (i.e., currently mounted) accessory andthe previously mounted accessory(s). The mounting number of accessoriesmay indicate the number of mounting portions on the robot 500. Forexample, one or more accessories may be mounted on each of the head part(A) and the body part (B) of the robot 500.

In various embodiments, the processor 550 may classify thecharacteristics of accessories into stages (e.g., first stage, secondstage, and third stage). In an embodiment, such stages for classifyingthe characteristics of the accessories may be determined, for example,based on the number and/or positions of mounted accessories. In anembodiment, when the type of a mounted accessory is determined, therobot 500 may determine a modeling rule (e.g., a face element (e.g., abear face, a rabbit face, a cartoon character face, a dinosaur face)associated with to a specific character assigned to the accessory)corresponding to the accessory type and provide a change in facialexpression stage by stage, based on the modeling rule depending on thenumber of mounted accessories.

According to an embodiment, as shown in Table 3 below, thecharacteristics of accessories may be classified according to the numberof mounted accessories into a first stage (e.g., single), a second stage(e.g., from two to N−1), and a third stage (e.g., N).

TABLE 3 Mounting Stage number 1^(st) category 2^(nd) category 3 ^(rd)category 1^(st) 1 Change robot Provide additional Provide properties atevent-related robot connected minimum level properties function 2^(nd)From 2 to Change robot Provide deepen Provide N − 1 properties atevent-related robot connected intermediate properties functions in level3 ^(rd) N Change robot combination or properties at independentlymaximum level

In an embodiment, the characteristics of accessories may bedifferentially provided stage by stage (e.g., first, second, and thirdstages) and differently applied according to the categories of theaccessories (e.g., first, second, and third categories). For example,the accessories of the first category may be classified into the first,second, and third stages according to the mounting number, and theaccessories of the second or third category may be classified into thefirst and second stages according to the mounting number. Thisclassification is exemplary only.

As shown in Table 3, the accessories of the first category may bedifferently applied to change the properties of the robot 500, dependingon the mounting number thereof.

When a single accessory of the first category is mounted (i.e., firststage), this accessory may be applied to change some properties (e.g.,facial expressions, sounds, etc.) of the robot 500 at the minimum level.That is, the default properties of the robot 500 may be changedaccording to a low degree of completion of the accessory characteristics(e.g., a neutral expression or happy expression and a related sound).

When two or more (smaller than N) accessories of the first category aremounted (i.e., second stage), these accessories may be applied to changemuch more properties (e.g., facial expressions, sounds, motions,lighting, etc.) of the robot 500 at the intermediate level. That is, thedefault properties and previously changed properties of the robot 500may be changed according to a middle degree of completion of theaccessory characteristics.

When all mountable accessories of the first category are mounted (i.e.,third stage), these accessories may be applied to change wholeproperties (e.g., facial expressions, sounds, motions, lighting, UXfeatures, skin color/patterns, personality, etc.) of the robot 500 atthe maximum level. That is, the default properties and previouslychanged properties of the robot 500 may be changed according to a highdegree of completion of the accessory characteristics.

As shown in Table 3, the accessories of the second category may bedifferently applied to change the properties of the robot 500, dependingon the mounting number thereof.

When a single accessory of the second category is mounted (i.e., firststage), this accessory may be applied to provide additionalevent-related properties (e.g., facial expressions, sounds, etc.) of therobot 500. That is, in addition to the default properties of the robot500, additional robot properties associated with the event of theaccessory may be provided. For example, if the event of the accessory isChristmas, a Christmas carol may be additionally provided to theproperties of the robot 500.

When two or more (equal to or smaller than N) accessories of the secondcategory are mounted (i.e., second or third stage), these accessoriesmay be applied to provide deepen event-related properties (e.g., facialexpressions, sounds, motions, lighting, UX features, skincolor/patterns, personality, etc.) of the robot 500. That is, inaddition to the default properties of the robot 500 and previously addedproperties, deepen (or detailed) robot properties associated with theevent of the accessory may be further provided. For example, in additionto providing a Christmas carol, a facial expression of a snowman, asound-related motion (e.g., dancing), and lighting (e.g., twinklelighting to a carol song) may be further provided.

As shown in Table 3, the accessories of the third category may bedifferently applied to change the properties of the robot 500, dependingon the mounting number thereof.

When a single accessory of the third category is mounted (i.e., firststage), this accessory may be applied to provide a connected function tothe robot 500. That is, in addition to the default properties of therobot 500, a function of the accessory may be provided as an additionalrobot function. For example, if the mounted accessory is a projector, animage output function of the projector may be added to the robot 500.

When two or more (equal to or smaller than N) accessories of the thirdcategory are mounted (i.e., second or third stage), these accessoriesmay be applied to provide connected functions in combination orindependently to the robot 500. That is, in addition to the defaultproperties of the robot 500 and previously added functions, additionalor extended functions of the accessories may be further provided. Forexample, in addition to the image output function of the projector, acontent output function of a tablet PC and a function of outputtingrelated information (e.g., motion, sound, gesture, reaction, etc.)synchronized with the content may be added to the robot 500.

According an embodiment, in connection with Table 3, the maximum numberof mountable accessories may be varied according to the type ofaccessories. For example, when the accessory type is a rabbit, up tothree accessories such as ear, tail, and arm may be mounted. In anotherexample, when the accessory type is a dinosaur, up to four accessoriessuch as horn, teeth, tabula, and tail may be mounted.

According to an embodiment, applying the above-mentioned stages may bevaried according to the total number (N) of mountable accessories. Forexample, if the total number of mountable accessories (in case ofcharacter-type accessories) is four, the third stage may be applied whenfour accessories are mounted as shown in Table 3. However, if the totalnumber of mountable accessories (in case of character-type accessories)is two, the third stage may be applied even if only two accessories aremounted.

According to embodiments, as shown in Tables 1, 2, and 3, different UXfeatures (e.g., display theme, face, sound, voice, motion, haptic,lighting, color, etc.) may be changeably applied depending on the typesand number of mounted accessories. In an embodiment, such UX featuresexpressible through the robot 500 may be also selectively appliedaccording to the available components of the robot 500. In anembodiment, for example, expressible sounds may include inherent soundsof the accessory (e.g., character-specific or role-specific sound suchas police) and feedback sounds corresponding to a user input (e.g.,touch, hug, stroke, etc.) for interacting with the robot 500.

According to various embodiments, at operation 930, the processor 550may identify the mounting position of the accessory, as follows.

In an embodiment, when the mounting of a certain accessory is detected,the processor 550 may identify the mounting position of the detected(i.e., currently mounted) accessory. At this time, the processor 550 mayidentify the mounting positions in consideration of all of the currentlymounted accessory and previously mounted accessories. The mountingpositions of accessories may be variously distributed over the head part(A) and the body part (B) as described above with reference to FIG. 6 .When identifying the mounting position of the accessory, the processor550 may determine whether the accessory is mounted at a correct positionor a wrong position, and thus provide a relevant feedback.

FIG. 10 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure.

Referring to FIG. 10 , at operation 1001, the processor 550 of the robot500 may process actions corresponding to a default operation state ofthe robot 500. In an embodiment, the processor 550 may express a facialexpression of the robot 500 or output a sound of the robot 500, based ondefault properties which have been set in advance in the robot 500. Thedefault operation state may refer to a state where no accessory ismounted on the robot 500.

At operation 1003, the processor 550 may determine (or monitor) whethermounting of a first accessory is detected in the default operation stateof the robot 500. In an embodiment, the first accessory may be a certainsingle accessory or a certain pair of accessories (e.g., ears).

When the mounting of the first accessory is not detected at theoperation 1003, the processor 550 may return to the operation 1001.

When the mounting of the first accessory is detected at the operation1003, the processor 550 may identify the accessory characteristics ofthe first accessory at operation 1005, based on the type, number, and/orposition of the first accessory(s). That is, when the mounting of thefirst accessory(s) is detected, the processor 550 may identify the typeof the first accessory(s), the number of the first accessory(s), and/orthe mounting position of the first accessory(s).

At operation 1007, the processor 550 may apply the accessory mountingcharacteristics of a first stage. In an embodiment, the processor 550may apply the identified accessory characteristics to the defaultproperties of the robot 500 as described above in Table 3 and thenprovide a corresponding feedback with expressiveness of the first stage.In an embodiment, when the mounting position of the first accessory isnot proper, that is, when the first accessory is mounted at a wrongposition (e.g., when a tail accessory is mounted at an ear-mountableposition), the processor 550 may provide a relevant feedback (e.g., analert of mounting error).

Then, at operation 1009, the processor 550 may determine (or monitor)whether mounting of a second accessory is detected. In an embodiment,the second accessory may be a certain single accessory or a certain pairof accessories (e.g., arms). The type of the second accessory(s) may beidentical to or different from that of the first accessory(s) mountedpreviously.

When the mounting of the second accessory is not detected at theoperation 1009, the processor 550 may return to the operation 1007. Inan embodiment, after a certain accessory is mounted, the processor 550may determine (or monitor) whether the mounted accessory is detached. Ifso, the processor 550 may offer a relevant feedback.

When the mounting of the second accessory is detected at the operation1009, the processor 550 may identify the accessory characteristics ofthe second accessory at operation 1011, based on the type, number,and/or position of the second accessory(s). That is, when the mountingof the second accessory(s) is detected, the processor 550 may identifythe type of the second accessory(s), the number of the secondaccessory(s), and/or the mounting position of the second accessory(s).

At operation 1013, the processor 550 may apply the accessory mountingcharacteristics of a second stage. In an embodiment, the processor 550may apply the identified accessory characteristics to the properties ofthe robot 500 to which the accessory characteristics of the first stagehave been applied as described above in Table 3, and provide acorresponding feedback with expressiveness of the second stage. In anembodiment, when the type of the second accessory is different from thatof the first accessory mounted previously (e.g., when a rabbit earaccessory is mounted after a bear tail accessory has been mounted), theprocessor 550 may provide a relevant feedback.

Then, at operation 1015, the processor 550 may determine (or monitor)whether mounting of an N-th accessory is detected. In an embodiment, theN-th accessory may be a third accessory or the last accessory among allmountable accessories.

When the mounting of the N-th accessory is not detected at the operation1015, the processor 550 may return to the operation 1013. In anembodiment, after a certain accessory is mounted, the processor 550 maydetermine (or monitor) whether the mounted accessory is detached. If so,the processor 550 may offer a relevant feedback.

When the mounting of the N-th accessory is detected at the operation1015, the processor 550 may identify the accessory characteristics ofthe N-th accessory at operation 1017, based on the type, number, and/orposition of the N-th accessory(s). That is, when the mounting of theN-th accessory(s) is detected, the processor 550 may identify the typeof the N-th accessory(s), the number of the N-th accessory(s), and/orthe mounting position of the N-th accessory(s).

At operation 1019, the processor 550 may apply the accessory mountingcharacteristics of a third stage. In an embodiment, the processor 550may apply the identified accessory characteristics to the properties ofthe robot 500 to which the accessory characteristics of the first andsecond stages have been applied as described above in Table 3, andprovide a corresponding feedback with expressiveness of the third stage.

According to an embodiment, whenever any accessory is mounted, theprocessor 550 may determine whether the mounted accessory is the lastmountable accessory. This may be required because the maximum number ofmountable accessories may be varied according to the type of accessoriesas described above. For example, when the accessory type is a rabbit, upto three accessories such as ear, tail, and arm may be mounted. Inanother example, when the accessory type is a dinosaur, up to fouraccessories such as horn, teeth, tabula, and tail may be mounted.

According to an embodiment, applying the above-mentioned stages may bevaried according to the total number (N) of mountable accessories. Forexample, if the total number of mountable accessories (in case ofcharacter-type accessories) is four, the third stage may be applied whenfour accessories are mounted as shown in Table 3. However, if the totalnumber of mountable accessories (in case of character-type accessories)is two, the third stage may be applied even if only two accessories aremounted.

FIG. 11 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure.

Referring to FIG. 11 , at operation 1101, the processor 550 of the robot500 may detect mounting of an accessory. In an embodiment, the robot 500may be in a default operation state (i.e., a state where the robot 500is operating according to default properties which have been set inadvance) or in a particular operation state where the robot 500 isoperating according to properties changed by the characteristics ofpreviously mounted accessory(s). In an embodiment, the processor 550 maydetect the mounting of the accessory, based on a change in signal (e.g.,a magnetic field) between the sensor circuit 703 (e.g., Hall sensor) ofthe robot 500 and the sensor module 725 of the accessory.

According to an embodiment, the sensor module 725 may include a modulefor generating a signal for detecting the mounting position of theaccessory 600 in the robot 500. For example, the sensor module 725 mayinclude a magnetic object (e.g., a magnet or a rubber magnet) thataffects current. In an embodiment, the sensor module 725 may operate asa mounting member. For example, when the sensor module 725 isimplemented as a magnet (or rubber magnet), it may be magneticallyattached to the mounting member (e.g., iron) provided at anaccessory-mountable position on the housing of the robot 500.

When the mounting of the accessory is detected, the processor 550 mayacquire accessory information from the mounted accessory at operation1103. According to an embodiment, the processor 550 may acquire theaccessory information by communicating with the communication module 723of the accessory through the communication circuit 705 (e.g., circuitfor contactless communication) established for communication (orwireless connection) with the accessory. For example, the contactlesscommunication may be implemented with at least one contactless proximitycommunication technique such as NFC, RFID communication, or MSTcommunication.

In an embodiment, the accessory 600 and the robot 500 may communicatewith each other through any other wireless communication (e.g., Wi-Fi,Bluetooth communication) in addition to the contactless communication.In an embodiment, the communication module 723 of the accessory may beimplemented with an NFC tag. For example, the accessory may input andstore information about the accessory (e.g., an accessory uniqueidentifier) in the NFC tag, and the processor 550 may read informationstored in the NFC tag through the communication circuit 705. Acquiringthe accessory information may include, for example, acquiring accessoryinformation of the detected (or currently mounted) accessory and/or ofthe previously mounted accessory(s).

At operation 1105, the processor 550 may identify accessorycharacteristics. According to an embodiment, the processor 550 mayidentify an accessory type, the number of mounted accessories, and aposition of the mounted accessory. In an embodiment, in response to themounting of the accessory, the processor 550 may identify the accessorytype (e.g., a tail of rabbit, tabula of dinosaur, etc.) from theacquired accessory information (e.g., a unique identifier), identify thenumber of mounted accessories from the acquired accessories, andidentify the accessory mounting position by identifying the location ofthe sensor circuit 703 that detects the accessory.

At operation 1107, the processor 550 may determine whether the accessoryis appropriately mounted. According to an embodiment, based on theacquired accessory information and information about the correspondingmounting position, the processor 550 may determine whether the accessorymounting position is suitable. Further, based on the accessoryinformation of the detected (or currently mounted) accessory andinformation of the previously mounted accessory, the processor 550 maydetermine whether both accessories have the same type.

When it is determined at the operation 1107 that the accessory mountingposition is suitable, at operation 1109, the processor 550 may apply theproperties of the robot 500 based on the identified accessorycharacteristics. According to an embodiment, based on the identifiedaccessory characteristics, the processor 550 may determine theproperties of the robot 500 for providing a feedback corresponding tothe characteristics of the accessory through the robot 500. In anembodiment, the processor 550 may determine the properties of the robot500 as the characteristics of the accessory, for example, animal,dinosaur, plant, insect, occupation (e.g., police, doctor, scientist,firefighter, sports athlete), character (e.g., cartoon character, angel,demon), linked device function, or the like.

Then, at operation 1111, the processor 550 may provide a first feedbackbased on the properties of the robot 500. According to an embodiment,the processor 550 may control at least one component to output thefeedback in accordance with the determined properties of the robot 500.For example, depending on the properties of the robot 500, the processor550 may provide at least one feedback (e.g., a change in facialexpression, a change in motion, a change in sound (or voice), lighting,and/or the like) to the user through an output module (e.g., the outputmodule 707 in FIG. 7 ). The feedback(s) may be selected according tosuitable components of the robot 500.

When it is determined at the operation 1107 that the accessory mountingposition is not suitable, at operation 1113, the processor 550 may notapply the properties of the robot 500 based on the identified accessorycharacteristics. According to an embodiment, when the accessory mountingposition is not proper, that is, when the accessory is mounted at awrong position (e.g., when a tail accessory is mounted at anear-mountable position), the processor 550 may provide a relevantfeedback (e.g., an alert of mounting error). In an embodiment, when thetype of the accessory is different from that of the previously mountedaccessory (e.g., when a rabbit ear accessory is mounted after a beartail accessory has been mounted), the processor 550 may provide arelevant feedback.

Then, at operation 1115, the processor 550 may provide a secondfeedback. According to an embodiment, the processor 550 may provide thesecond feedback informing a mounting error based on at least one of avisual element, an auditory element, or a tactile element.

FIG. 12 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure.

Referring to FIG. 12 , at operation 1201, the processor 550 of the robot500 may perform a particular operation of the robot 500 based on setproperties. In an embodiment, the robot 500 may be in a defaultoperation state (i.e., a state where the robot 500 is operatingaccording to default properties which have been set in advance) or in aparticular operation state where the robot 500 is operating according toproperties changed by the characteristics of previously mountedaccessory(s).

At operation 1203, the processor 550 may determine (or monitor) whethera set input is detected. According to an embodiment, the set input mayinclude, for example, an external input or an internal input. Theexternal input may include a user input (e.g., touch, hug, stroke, etc.)or an input device (e.g., a device power-on, a content execution, etc.)of a mounted accessory (e.g., an external device or a function-connecteddevice). For example, the user input may include a user interaction suchas pressing the body part (B) of the robot 500 or stroking the head part(A) of the robot 500. The internal input may include an input based oncontext awareness of the processor 550. For example, the processor 550may recognize, as an input, a change in surrounding environments (e.g.,a change in a time zone or a change in a place).

When no input is detected at the operation 1203, the processor 550 mayreturn to the operation 1201.

When the set input is detected at the operation 1203, the processor 550may provide a feedback corresponding to the detected input at operation1205, based on applied properties. According to an embodiment, theprocessor 550 may provide a feedback based on the robot 500 and/or theaccessory. For example, the processor 550 may provide at least onefeedback (e.g., change in facial expression, body twisting, bodybending, body shaking, change in sound or voice, change in color,lighting, etc.) to the user through the output module (e.g., the outputmodule 707 in FIG. 7 ) of the robot 500. In another example, theprocessor 550 may provide at least one feedback (e.g., lighting,vibration, or color change, etc.) to the user through the mountedaccessory.

FIG. 13 is a flow diagram illustrating a method for operating a robotaccording to an embodiment of the disclosure.

Referring to FIG. 13 , at operation 1301, the processor 550 of the robot500 may detect mounting of an accessory. In an embodiment, the robot 500may be in a default operation state (i.e., a state where the robot 500is operating according to default properties which have been set inadvance) or in a particular operation state where the robot 500 isoperating according to properties changed by the characteristics ofpreviously mounted accessory(s). In an embodiment, the processor 550 maydetect the mounting of the accessory, based on a change in signal (e.g.,a magnetic field) between the sensor circuit 703 (e.g., Hall sensor) ofthe robot 500 and the sensor module 725 of the accessory.

At operation 1303, the processor 550 may identify accessorycharacteristics. According to an embodiment, the processor 550 mayidentify an accessory type, the number of mounted accessories, and aposition of the mounted accessory.

At operation 1305, the processor 550 may determine applicableexpressiveness based on the mounted accessory (e.g., additionalexpressive capabilities, functions, or service levels by the furthermounting (or stage change) of the accessory). According to anembodiment, in case of the accessory of the first category, theprocessor 550 may check whether there are additionally operablefunctions, additionally changeable facial expressions and/or voices,additional events, or additional personalities, depending on the furthermounted accessory(s).

At operation 1307, the processor 550 may determine an expression rangebased on the determined expressiveness. According to an embodiment, theprocessor 550 may determine further particular actions or functionsapplicable through the robot 500 according to the determinedexpressiveness or service level.

At operation 1309, the processor 550 may perform at least one operationbased on the determined expression range. According to an embodiment,the processor 550 may control components (e.g., the sensor circuit, thecommunication circuit, the speaker, the output device, etc.)corresponding to the further applicable actions or functions inaccordance with the expressiveness and also control the output module707 for outputting the corresponding stage of the accessorycharacteristics. For example, when an additional function is a lightingfunction, the processor 550 may recognize a user input (e.g., touch),illumination, or set (or received) notification to perform a lightingcontrol. In another example, when an addition function is a certainevent, the processor 550 may perform playback of event-related content,apply (or change) a color (or property) corresponding to the event, orcontrol an event-related motion to express event features. In stillanother example, in order to execute a function interoperable with theaccessory (e.g., a function-connected device), the processor 550 mayperform interactions with the accessory, perform synchronization forinterworking with content, or recognize a user input (e.g., touch) onthe robot 500 or the accessory.

At operation 1311, the processor 550 may provide at least one feedbackbased on a result of performing operations. For example, the processor550 may provide a lighting feedback (e.g., applying a color, a blinkingeffect, etc.), a mood lighting mode, a lighting feedback in response toa notification (e.g., a change in color and/or blink pattern accordingto notification information such as a phone or message). In anotherexample, the processor 550 may provide an additional event-relatedfeedback (e.g., a change of voice tone according to applyingpersonality). In still another example, the processor 550 may provide afeedback by adding or extending a function based on interworking withthe accessory, or provide a feedback corresponding to a user interactionwith the accessory.

FIG. 14 is a diagram illustrating examples of an accessory mounted on arobot according to an embodiment of the disclosure.

Referring to FIG. 14 , it shows three examples in which at least oneaccessory of the first category (i.e., character type) as describedabove in Tables 1, 2 and 3 is mounted on the robot 500.

Referring to FIG. 14 , the accessories of the first category may includecharacter-type accessories that represent user-friendly characters suchas, for example, animal, dinosaur, plant, insect, occupation (or role,e.g., police, doctor, scientist, firefighter, sports athlete), cartooncharacter, game character, angel, demon, and the like. For example, FIG.14 shows a first accessory 1410 expressing the characteristics of abear, a second accessory 1420 expressing the characteristics of arabbit, and a third accessory 1430 expressing the characteristics of adinosaur. Although FIG. 14 shows ear-type accessories expressing therespective characteristics, other type accessories such as arms, a tail,or a horn may be used for expressing the corresponding accessorycharacteristics.

According to an embodiment, when the characteristics of the accessory ofthe first category is applied, a default facial expression of the robot500 provided through a display device 1400 of the robot 500 may bechanged to a facial expression corresponding to the applied accessorycharacteristics. For example, when the first accessory 1410 is mounted,the default facial expression of the robot 500 may be changed to afacial expression of a bear corresponding to the characteristics of thefirst accessory 1410. When the second accessory 1420 is mounted, thedefault facial expression of the robot 500 may be changed to a facialexpression of a rabbit corresponding to the characteristics of thesecond accessory 1420. When the third accessory 1430 is mounted, thedefault facial expression of the robot 500 may be changed to a facialexpression of a dinosaur corresponding to the characteristics of thethird accessory 1430.

In addition, although not shown, when the accessory is mounted, a sound(e.g., voice) corresponding to the characteristics of the accessory maybe also changed together with a facial expression corresponding to thecharacteristics of the accessory.

According to an embodiment, two or more accessories having differentcharacteristics (e.g., a first character accessory and a secondcharacter accessory) may be mounted in combination. In this case, forexample, the robot 500 may apply such different characteristics of theaccessories in the order of priority corresponding to the order ofmounting accessories. In another example, the robot 500 may apply suchdifferent characteristics of the accessories in the order of prioritydepending on mounting regions (e.g., applying first a facial expressionaccording to the characteristics of the accessory mounted on the headpart (A)).

According to an embodiment, different-type accessories may be mounted onthe robot 500 in combination. In this case, for example, the processor550 may apply a facial expression according to the characteristics ofone accessory mounted on the head part (A), and also apply a motionaccording to the characteristics of another accessory mounted on thebody part (B). In another example, when different-type accessories aremounted together in the same region (e.g., the head part (A) or the bodypart (B)), the processor 550 may preferentially apply thecharacteristics of the accessory firstly mounted on that region. Thepriority of applying the characteristics may be set by the user.

FIG. 15 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 15 , FIG. 15 shows examples of a change in expressionstage by stage depending on the number of accessories mounted on therobot 500.

In an embodiment, the characteristics of accessories may bedifferentially provided stage by stage (e.g., first, second, and thirdstages) and differently applied according to the categories of theaccessories (e.g., first, second, and third categories). For example,the accessories of the first category may be classified into the first,second, and third stages according to the mounting number, and theaccessories of the second or third category may be classified into thefirst and second stages according to the mounting number. Thisclassification is exemplary only.

In FIG. 15 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 1500 and a defaultsound).

Example (B) shows that a first accessory 1510 (e.g., ears) having thecharacteristics of a bear is mounted on the head part (A) of the robot500. As shown, when the first accessory 1510 is mounted, the defaultfacial expression 1500 of the robot 500 may be changed to a facialexpression 1550 corresponding to the characteristics of a bear. Inaddition, a certain sound corresponding to the characteristics of a bearmay be outputted. According to an embodiment, when a single accessory(e.g., the first accessory 1510) is mounted (i.e., first stage), thisaccessory may be applied to change some properties (e.g., facialexpressions, sounds, etc.) of the robot 500 at the minimum level. Thatis, the default properties of the robot 500 may be changed according toa low degree of completion of the accessory characteristics (e.g., aneutral expression or happy expression and a related sound).

Example (C) shows that a second accessory 1520 (e.g., a tail) having thecharacteristics of a bear is mounted on the body part (B) of the robot500 after the first accessory 1510 (e.g., ears) having thecharacteristics of a bear is mounted. As shown, when the secondaccessory 1520 is further mounted, a specific motion corresponding tothe characteristics of a bear may be provided together with a relevantsound. According to an embodiment, when two or more (smaller than N)accessories (e.g., the first and second accessories 1510 and 1520) aremounted (i.e., second stage), these accessories may be applied to changemuch more properties (e.g., facial expressions, sounds, motions,lighting, etc.) of the robot 500 at the intermediate level. That is, thedefault properties and previously changed properties of the robot 500may be changed according to a middle degree of completion of theaccessory characteristics.

Example (D) shows that a third accessory 1530 (e.g., arms) having thecharacteristics of a bear is mounted on the body part (B) of the robot500 after the first and second accessories 1510 and 1520 (e.g., ears anda tail) having the characteristics of a bear are mounted. As shown, whenthe third accessory 1530 is further mounted, that is, when allaccessories for expressing the characteristics of a bear are mounted,full expressions may be provided. According to an embodiment, when allmountable accessories (e.g., the first, second and third accessories1510, 1520 and 1530) are mounted (i.e., third stage), these accessoriesmay be applied to change whole properties (e.g., facial expressions,sounds, motions, lighting, UX features, skin color/patterns,personality, etc.) of the robot 500 at the maximum level. That is, thedefault properties and previously changed properties of the robot 500may be changed according to a high degree of completion of the accessorycharacteristics.

FIG. 16 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 16 , FIG. 16 shows examples of a change in expressionstage by stage when respective accessories are mounted on or detachedfrom the robot 500.

In FIG. 16 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 1600 and a defaultsound).

Example (B) shows that a pair of first accessories 1610 (e.g., a rightear accessory 1611 and a left ear accessory 1612) having specificcharacteristics is being mounted on the head part (A) of the robot 500.As shown, while the first accessories 1610 are being mounted on therobot 500, no feedback is provided (e.g., the default facial expression1600 of the robot 500 is maintained). When one (e.g., the right earaccessory 1611) of the pair of the first accessories 1610 is completelymounted, a predetermined feedback (e.g., lighting through the right earaccessory 1611) may be provided. However, in case of the pair of thefirst accessories 1610, the default properties (e.g., the default facialexpression 1600 and the default sound) of the robot 500 may bemaintained even if only one (e.g., the right ear accessory 1611) of bothaccessories is mounted.

Example (C) shows that the pair of the first accessories 1610 (e.g., theright ear accessory 1611 and the left ear accessory 1612) is completelymounted on the head part (A) of the robot 500. At this time, apredetermined feedback (e.g., lighting such as twinkle and related soundinforming successful mounting) may be provided through the pair of thefirst accessories 1610. In addition, the default facial expression 1600of the robot may be changed to a facial expression 1650 corresponding tothe characteristics of the first accessories 1610, and also a relatedsound may be provided.

Example (D) shows that a second accessory 1620 (e.g., a tail accessory)is mounted on the body part (B) of the robot 500 after the firstaccessories 1610 are mounted. As shown, when the second accessory 1620is further mounted, a specific motion corresponding to thecharacteristics of the second accessory 1620 may be provided. Inaddition, a predetermined feedback (e.g., lighting such as twinkleand/or vibration) may be provided through the second accessory 1620.

Example (E) shows that the second accessory 1620 (e.g., a tailaccessory) is detached from the robot 500 after the first and secondaccessories 1610 and 1620 are mounted. At this time, a feedbackassociated with detachment (e.g., a motion of shaking a head from sideto side, and/or a related sound) may be provided.

Example (F) shows that one (e.g., the left ear accessory 1612) of thepair of the first accessories 1610 (e.g., the right ear accessory 1611and the left ear accessory 1612) is detached from the robot 500. At thistime, the changed properties of the robot 500 (e.g., the changed facialexpression 1650 and related sound) may be maintained. When the pair ofthe first accessories 1610 is completely detached, the changedproperties of the robot 500 (e.g., the changed facial expression 1650and related sound) may be return to the default properties (e.g., thedefault facial expression 1600 and the default sound) of the robot 500.

FIG. 17 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure.

Referring to FIG. 17 , FIG. 17 shows an example of providing a feedbackaccording to an interaction with a user, based on the characteristics ofthe accessory mounted on the robot 500.

In FIG. 17 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 1700 and a defaultsound).

Examples (B) and (C) show that a pair of first accessories 1710 (e.g., aright ear accessory and a left ear accessory) having specificcharacteristics is being sequentially mounted on the head part (A) ofthe robot 500.

As shown in example (B), while the first accessories 1710 are beingmounted on the robot 500, no feedback is provided (e.g., the defaultfacial expression 1700 of the robot 500 is maintained). When one (e.g.,the right ear accessory) of the pair of the first accessories 1710 iscompletely mounted, a predetermined feedback (e.g., lighting through theright ear accessory) may be provided. However, in case of the pair ofthe first accessories 1710, the default properties (e.g., the defaultfacial expression 1700 and the default sound) of the robot 500 may bemaintained even if only one (e.g., the right ear accessory) of bothaccessories is mounted.

Example (C) shows that the pair of the first accessories 1710 (e.g., theright ear accessory and the left ear accessory) is completely mounted onthe head part (A) of the robot 500. At this time, a predeterminedfeedback (e.g., lighting and related sound informing successfulmounting) may be provided through the first accessories 1710. Inaddition, the default facial expression 1700 of the robot may be changedto a facial expression 1750 corresponding to the characteristics of thefirst accessories 1710, and also a related sound may be provided.

Example (D) shows an interaction with the user. As shown, when a userinput 1770 (e.g., a motion of sticking the body part (B)) is detected, afeedback corresponding to the user input 1770 may be provided. Forexample, the facial expression 1750 and sound being provided accordingto the characteristics of the first accessories 1710 may be changed toanother facial expression 1760 and related sound in response to the userinput 1770. In an embodiment, a change in facial expression may beprovided according to a modeling rule (e.g., a face element (e.g., abear face, a rabbit face, a cartoon character face, a dinosaur face)associated with a specific character assigned to the accessory)corresponding to the accessory characteristics.

FIG. 18 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure.

Referring to FIG. 18 , FIG. 18 shows an example of providing a feedbackaccording to an interaction with a user, based on the characteristics ofthe accessory mounted on the robot 500. Especially, FIG. 18 shows anexample of providing a plurality of feedbacks such as haptic and soundfeedbacks in response to user interactions.

In FIG. 18 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 1800 and a defaultsound).

Example (B) shows that a pair of first accessories 1810 (e.g., earaccessories) having specific characteristics are mounted on the headpart (A) of the robot 500. As shown, the default facial expression 1800of the robot 500 may be changed to a facial expression 1850corresponding to the characteristics of the first accessories 1810, andalso a related sound may be provided.

Example (C) shows an interaction with the user. As shown, when a firstuser input 1870 (e.g., a motion of stroking the head part (A)) and asecond user input 1880 (e.g., a motion of hugging the body part (B)) aredetected, a plurality of feedbacks corresponding to the first and seconduser inputs 1870 and 1880 may be provided. For example, the facialexpression 1850 and sound being provided according to thecharacteristics of the first accessories 1810 may be changed to anotherfacial expression 1860 and related sound in response to the first andsecond user inputs 1870 and 1880. In addition, the robot 500 may providea haptic feedback and a sound feedback through the head part (A) inresponse to the first user input 1870 (e.g., stroke) and also provide ahaptic feedback through the body part (B) in response to the second userinput 1880 (e.g., hug). In an embodiment, a change in facial expressionmay be provided according to a modeling rule (e.g., a face element(e.g., a bear face, a rabbit face, a cartoon character face, a dinosaurface) associated with a specific character assigned to the accessory)corresponding to the accessory characteristics.

FIG. 19 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure.

Referring to FIG. 19 , FIG. 19 shows an example of providing a feedbackthrough context awareness, based on the characteristics of the accessorymounted on the robot 500. That is, FIG. 19 shows an example of sensing asurrounding environment (or context awareness) and providing acorresponding feedback such as a lighting control.

In FIG. 19 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 1900 and a defaultsound).

Example (B) shows that first accessories 1910 having specificcharacteristics are mounted on the robot 500. As shown, the defaultfacial expression 1900 of the robot 500 may be changed to a facialexpression 1950 corresponding to the characteristics of the firstaccessories 1910, and also a related sound may be provided.

Example (C) shows an example of sensing a surrounding environment (orcontext awareness) and providing a corresponding feedback. For example,the robot 500 may sense the ambient illumination through theillumination sensor and determine whether the ambient illuminationsatisfies a predetermined condition (e.g., a given illumination level orless). If so, the robot 500 may provide a feedback of controlling alighting function of the first accessory 1910 to express a sleep modesuch as a mood light (e.g., yellow lighting) and also provide a sleepingfacial expression 1970.

FIG. 20 is a diagram illustrating an operating example of a robot basedon a mounted accessory according to an embodiment of the disclosure.

Referring to FIG. 20 , FIG. 20 shows an example of providing a feedbackthrough a recognized notification, based on the characteristics of theaccessory mounted on the robot 500. That is, FIG. 20 shows an example ofrecognizing a notification (e.g., receiving a call or message) andproviding a corresponding feedback such as a lighting control.

In FIG. 20 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression 2000 and a defaultsound).

Example (B) shows that first accessories 2010 having specificcharacteristics are mounted on the robot 500. As shown, the defaultfacial expression 2000 of the robot 500 may be changed to a facialexpression 2050 corresponding to the characteristics of the firstaccessories 2010, and also a related sound may be provided.

Example (C) shows an example of recognizing a notification (e.g.,receiving a call or message) and providing a corresponding feedback. Forexample, the robot 500 may provide a feedback of controlling a lightingfunction of the first accessory 2010 to express a notification through acolor change (e.g., green lighting) of the first accessory 2010.

FIG. 21 is a diagram illustrating examples of an accessory mounted on arobot according to an embodiment of the disclosure.

Specifically, FIG. 21 shows various examples in which at least oneaccessory of the second category (i.e., event type) as described abovein Tables 1, 2 and 3 is mounted on the robot 500.

Referring to FIG. 21 , the accessory(s) of the second category mayinclude event-type accessories (e.g. hat, clothing, cloak, tie,necklace, glasses, stethoscope, etc.) associated with a particular event(e.g., Christmas, Halloween, birthday, country-specific costume,specific effect (e.g., color variation, pattern variation, materialchange), etc.) or accessories for convenience (e.g., cup holders,storage boxes, etc.). According to an embodiment, FIG. 21 shows a firstaccessory 2110 expressing the characteristics of a first dress, a secondaccessory 2120 expressing the characteristics of a second dress, a thirdaccessory 2130 expressing the characteristics of a third dress, a fourthaccessory 2140 (e.g. police cap) and a fifth accessory 2150 (e.g. policevest) expressing the characteristics of a first occupation (e.g.police), and a sixth accessory 2160 (e.g., medical headlamp) and aseventh accessory 2170 (e.g. stethoscope) expressing the characteristicsof a second occupation (e.g., doctor).

When applying the characteristics of the accessory(s) of the secondcategory, the default properties (e.g., the default facial expressionand the default sound) of the robot 500 being provided through thedisplay device 1400 of the robot 500 may be maintained, and also anevent-specific facial expression and sound may be further provided inaccordance with the characteristics of the mounted accessory. Inaddition, depending on the number of mounted accessories, a personality(or individuality) may be further applied to provide deepenevent-related properties. For example, when the fourth and fifthaccessories 2140 and 2150 expressing the characteristics of a police aremounted, the personality of the robot 500 may be changed to thepersonality of the police such as a strict facial expression and astrict voice tone.

FIG. 22 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 22 , FIG. 22 shows examples of a change in expressionstage by stage depending on the number of accessories mounted on therobot 500.

In an embodiment, the characteristics of accessories may bedifferentially provided stage by stage (e.g., first, second, and thirdstages) and differently applied according to the categories of theaccessories (e.g., first, second, and third categories).

In FIG. 22 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression and a defaultsound).

Example (B) shows that a first accessory 2210 (e.g., a police cap)having the specific characteristics (e.g., the characteristics of apolice) is mounted on the head part (A) of the robot 500. As shown, whenthe first accessory 2210 is mounted, a facial expression and soundcorresponding to the characteristics of the police may be furtherprovided. According to an embodiment, when the first accessory 2210 ismounted (i.e., first stage), the default properties of the robot 500 maybe maintained, and an additional facial expression and sound may befurther applied according to the characteristics of the first accessory2210.

Example (C) shows that a second accessory 2220 (e.g., a police signalrod) having the characteristics of the police is mounted on the bodypart (B) of the robot 500 after the first accessory 2210 (e.g., a policecap) having the same police characteristics is mounted. As shown, whenthe second accessory 2220 is further mounted, a specific lighting effect(e.g., signal rod lighting) may be provided through the second accessory2220 together with a relevant sound (e.g., siren). According to anembodiment, when the first and second accessories 2210 and 2220 aremounted together (i.e., second stage), these accessories may be appliedto provide deepen event-related properties (e.g., facial expressions,sounds, motions, lighting, UX features, skin color/patterns,personality, etc.) of the robot 500. That is, in addition to the defaultproperties of the robot 500 and previously added properties, deepen (ordetailed) robot properties associated with the event of the accessorymay be further provided.

FIG. 23 is a diagram illustrating examples of an accessory mounted on arobot according to embodiment of the disclosure.

Specifically, FIG. 23 shows various examples in which at least oneaccessory (e.g., an external device or a hardware device) of the thirdcategory (i.e., function-connected type) as described above in Tables 1,2 and 3 is mounted on the robot 500.

Referring to FIG. 23 , the accessory(s) of the third category mayinclude function-connected accessories for extending the functionalityof the robot 500 by combining (or functionally connecting) an externaldevice (for providing specific content and/or service) such as aprojector, a tablet PC, a speaker, an internet of thing (IoT) device, alighting device, or a camera. According to an embodiment, when theaccessory of the third category is mounted, this accessory may beapplied to provide a connected function to the robot 500. That is, inaddition to the default properties of the robot 500, a function of theaccessory may be provided as an additional robot function. For example,if the mounted accessory is a projector, an image output function of theprojector may be added to the robot 500. In an embodiment, when theaccessory of the third category is mounted, the robot 500 may perform aninteraction linked to the mounted accessory or a function ofinterworking with service content provided (or played) through themounted accessory.

According to an embodiment, FIG. 23 shows that a first accessory 2310having a first function, a second accessory 2320 having a secondfunction, and a third accessory 2330 having a third function are mountedon the robot 500.

When the characteristics of the accessory of the third category areapplied, the default properties (e.g., the default facial expression andthe default sound) of the robot 500 may be maintained. In addition, aspecific facial expression, sound, and/or function (or service)corresponding to the characteristics of the mounted accessory may befurther provided.

For example, when the first accessory 2310 is a tablet PC, the robot 500may perform a function of playing content (e.g., image or video) showingthe properties of the robot 500. In another example, when the secondaccessory 2320 is a projector, the robot 500 may perform a function ofprojecting content (e.g., image or video) showing the properties of therobot 500. In still another example, when the third accessory 2320 is aspeaker, the robot 500 may perform a role of disc jockey or perform asurround sound function. In yet another example, when the mountedaccessory is a lighting device, the robot 500 may perform a function ofa mood light. In further another example, when the mounted accessory isan IoT control device, the robot 500 may perform a controller forcontrolling IoT devices.

FIG. 24 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 24 , FIG. 24 shows examples of a change in expressionstage by stage depending on the number of accessories mounted on therobot 500.

In an embodiment, the characteristics of accessories may bedifferentially provided stage by stage (e.g., first, second, and thirdstages) and differently applied according to the categories of theaccessories (e.g., first, second, and third categories).

When a single accessory of the third category is mounted (i.e., firststage), this accessory may be applied to provide a connected function tothe robot 500. That is, in addition to the default properties of therobot 500, a function of the accessory may be provided as an additionalrobot function. For example, if the mounted accessory is a projector, animage output function of the projector may be added to the robot 500.

When two or more (equal to or smaller than N) accessories of the thirdcategory are mounted (i.e., second or third stage), these accessoriesmay be applied to provide connected functions in combination orindependently to the robot 500. That is, in addition to the defaultproperties of the robot 500 and previously added functions, additionalor extended functions of the accessories may be further provided. Forexample, in addition to the image output function of the projector, acontent output function of a tablet PC and a function of outputtingrelated information (e.g., motion, sound, gesture, reaction, etc.)synchronized with the content may be added to the robot 500.

In case of the second or third stage, the robot 500 may determinewhether to apply connected functions of the mounted accessories incombination or independently, and thereby provide a relevant feedback.

In FIG. 24 , example (A) shows a default state of the robot 500. In thedefault state, the robot 500 may be expressed according to defaultproperties thereof (e.g., a default facial expression and a defaultsound).

Example (B) shows that a first accessory 2410 (e.g., afunction-connected device such as a projector) having the specificcharacteristics (or function) is mounted on the body part (B) of therobot 500. As shown, when the first accessory 2410 is mounted, the robot500 may interoperate with the first accessory 2410 to provide contentplayback and interaction. According to an embodiment, when the firstaccessory 2410 is mounted (i.e., first stage), the function of the firstaccessory 2410 may be combined, added, or extended. For example, thedefault properties of the robot 500 may be maintained, and the functionof the first accessory 2410 (e.g., projecting content, being currentlyplayed by the robot, through the projector) may be further provided.

Example (C) shows that a second accessory 2420 (e.g., a speaker) ismounted on the body part (B) of the robot 500 after the first accessory2410 (e.g., a projector) is mounted. As shown, when the second accessory2420 is further mounted, the characteristics (or functions) of the firstand second accessories 2410 and 2420 may be combined, interworked, orapplied in extension.

In an embodiment, when the first and second accessories 2410 and 2420are mounted together (i.e., second stage), the robot 500 may determinewhether both accessories are applicable in combination. If so, the robot500 may provide additional functions by combining a content playbackfunction of the first accessory 2410 and a sound output function of thesecond accessory 2420.

FIG. 25 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 25 , FIG. 25 shows an example in which an accessory2500 (e.g., a tablet) is mounted on the robot 500, and the robot 500provides a feedback in response to a user input using the accessory2500. In addition, FIG. 25 shows an example of interaction between therobot 500 and the accessory 2500.

Example (A) shows an example in which the accessory 2500 (e.g., atablet) is mounted on the robot 500. When the accessory 2500 iscompletely mounted, the robot 500 may provide, as a feedback, a relatedsound (e.g., “Tablet is mounted!”). Also, the robot 500 may provide aspecific motion (e.g., a vertical swing motion of the head part (A)) asa feedback in response to the mounting of the accessory 2500.

When the accessory 2500 is mounted, the robot 500 may perform anoperation for interworking with the accessory 2500. Interworking betweenthe robot 500 and the accessory 2500 may be based on communicatingsignals with each other, for example, mechanically, electrically, or insoftware.

When the accessory 2500 (e.g., a function-connected device) is mounted,the robot 500 may perform interactions with the accessory 2500 in orderto execute a function interoperable with the accessory 2500, or performsynchronization for interworking with content played in the accessory2500. For example, the robot 500 may provide a complex function byinterlocking a function of the accessory 2500 with a function of therobot 500 (e.g., providing information (e.g., motion, sound, gesture,reaction, etc.) synchronized with content together with outputting thecontent at the tablet). The robot 500 may be in a state of interworkingwith the accessory 2500 and synchronized (e.g., time sync, audio sync,subtitle sync, etc.) with the content being played by the accessory2500.

Example (B) shows that the user executes content (or an application,etc.) at the accessory 2500, and the robot 500 provides a feedbackcorresponding to a user input for executing the content at the accessory2500. For example, the user may enter a user input (e.g., a tablettouch) for manipulating the accessory 2500 (e.g., playing content).Then, in response to the user input, the accessory 2500 may provideinput information (e.g., touch location, content information, etc.)related to the user input to the robot 500. Then, based on the receivedinput information, the robot 500 may recognize the user input andidentify a position (or direction) corresponding to the user input. In acertain embodiment, the robot 500 may include a camera module (notshown) or may be equipped with an accessory having a camera function. Inthis case, using the camera module or the mounted accessory having thecamera function, the robot 500 may recognize (or monitor) the user andprovide a relevant feedback.

As shown in Example (B), the robot 500 may provide a relevant feedbackin response to a user input entered into the accessory 2500. Forexample, the robot 500 may provide feedbacks such as a gaze change 2510in a direction corresponding to the user input and a related sound(“Here we go!”). For example, the robot 500 may provide a feedback suchas a glimpse of where the user has touched the accessory 2500.

Example (C) shows an example of providing a feedback corresponding to auser input while content is being executed through the accessory 2500.The user may enter a user input (e.g., a tablet touch) related to thecontent running through the accessory 2500. Then, in response to theuser input, the accessory 2500 may provide input information (e.g.,touch location, content information, etc.) related to the user input tothe robot 500. As shown, the user may enter a user input formanipulating (e.g., selecting) the content (e.g., text, image, video,etc.) being played in the accessory 2500.

In an embodiment, when there is a user input through the accessory 2500,the robot 500 may provide a feedback related to an object correspondingto the user input. For example, the robot 500 may provide a feedback(e.g., a sound) corresponding to the object (e.g., the text of thecaption (e.g., “mocking”)) in response to the user input. Also, therobot 500 may provide a feedback of a facial expression change 2520together with the sound feedback. For example, the robot 500 may read acaption of a place where the user touches through the accessory 2500.

Example (D) shows that the robot 500 interworking with the accessory2500 performs real-time sensing (e.g., context awareness) for asurrounding environment or user's expression or reaction and provide arelevant feedback. For example, when the user reacts (e.g., read along abook, follow a dance, follow a song, etc.) to content being executedthrough the accessory 2500, the robot 500 may recognize the user'sreaction and provide feedbacks such as a change in a facial expression2530 and a related sound (e.g., “Great!”). In a certain embodiment, therobot 500 may use a camera or a mounted accessory having a camerafunction to perform the real-time sensing, and provide a feedbackthrough image analysis.

In an embodiment, when the accessory 2500 (e.g., a function-connecteddevice) is mounted on the robot 500, the robot 500 may respond to theprogress of a content in the accessory 2500. For example, when the useris using the accessory 2500 (e.g., viewing video, or listening toaudio), the robot 500 may provide an emotional response associated withthe content of the accessory 2500 (e.g., a change in facial expressioncorresponding to a story or role of the content), provide a furtheraction to the user (e.g., quiz, game, read along, etc.), or provide aresponse based on the emotional state of the user (e.g., child) gettingbored (e.g., output a related sound (e.g., “Do you want to playsomething else?”).

FIG. 26 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 26 , FIG. 26 shows an example in which an accessory2600 (e.g., a tablet) is mounted on the robot 500, and the robot 500provides a feedback in response to a user input using the accessory2600. In addition, FIG. 26 shows an example of an empathic response to auser's reaction to the content of the accessory 2600 throughinterworking between the robot 500 and the accessory 2600.

Example (A) shows that the accessory 2600 (e.g., a tablet) is mounted onthe robot 500 and is in a state of waiting for content execution. Whenthe accessory 2600 is completely mounted, the robot 500 may provide, asa feedback, a related sound (e.g., “Tablet is mounted!”). Also, whenthere is no content execution by the accessory 2600 after the accessory2600 is mounted, the robot 500 may determine that the content executionis waiting, and thereby provide a feedback of a default facialexpression 2610. When a user input is entered into the accessory 2600,the robot 500 may recognize an action of executing content byinteracting with the accessory 2600.

Examples (B), (C), and (D) show that the user executes content (or anapplication, etc.) in the accessory 2600, and the robot 500 provides anempathic response to the executed content or a user's reaction. Forexample, the robot 500 may express an emotion linked to the learningcontent being executed in the accessory 2600 or provide an interactivelearning experience with the user.

As shown in Examples (B), (C), and (D), the robot 500 may providevarious feedback associated with an empathic response to the executedcontent or a user's reaction. For example, the robot 500 may performreal-time sensing (e.g., context awareness) for a surroundingenvironment or user's expression or reaction and thereby provide, asrelevant feedback, changes in facial expressions 2620, 2630, and 2640(e.g., a surprised or negative facial expression 2620, an excited facialexpression 2630, a happy facial expression 2640, etc.) and theirassociated sounds.

In an embodiment, the facial expression change may refer to a changefrom the default (e.g., neutral) facial expression 2610 to the otherfacial expression 2620, 2630, or 2640 corresponding to each situation,or a change from one facial expression (e.g., the surprised facialexpression 2620) according to the first response to another facialexpression (e.g., the happy facial expression 2640) according to thesecond response. In a certain embodiment, the robot 500 may use a cameraor a mounted accessory having a camera function to perform the real-timesensing, and provide a feedback through image analysis.

In an embodiment, when the accessory 2600 (e.g., a function-connecteddevice) is mounted on the robot 500, the robot 500 may sense a reactionof the user (e.g., the child) using the content, and provide a feedbackto empathize with the user's response. For example, the robot 500 mayprovide an empathic response (e.g., a laughing or crying sound) suitablefor the content of the accessory 2600.

FIG. 27 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 27 , FIG. 27 shows an example in which an accessory2700 (e.g., a tablet) is mounted on the robot 500, and the robot 500provides a feedback in response to a user input (e.g., touch or userreaction) using the accessory 2700.

Example (A) shows that the robot 500 provides a feedback correspondingto a user input entered into the accessory 2700 that is executingcontent. For example, the user may enter a user input (e.g., a tablettouch) into the accessory 2700, and the robot 500 may detect the userinput through the accessory 2700. Then, the robot 500 may provide arelated feedback in response to the user input. For example, the robot500 may provide feedbacks such as a gaze change 2710 in a directioncorresponding to the user input and a related sound (“Repeat!”).

In an embodiment, the robot 500 may provide a gaze change in real timein response to a change in the user input (e.g., a drag direction of atouch-and-drag input) in the accessory 2700. For example, the robot 500may provide a gaze of looking at a touch position in response to a userinput of touching the accessory 2700 and then provide a real-time gazechange in response to a touch movement (e.g., dragging). As such, therobot 500 may provide consecutive feedback that responds immediately touser inputs in the accessory 2700.

Example (B) shows an example of providing feedback in response tocontent or a user input. For example, the accessory 2700 may perform afunction such as stopping, playing, repeating, or terminating thecontent in response to a user input, and the robot 500 may providefeedback associated with a content-related control function in responseto the user input. For example, when the content is repeatedly played,the robot 500 may provide feedback such as a changed facial expression2720 (e.g., returning to the original gaze position of example (A) and arelated sound (e.g., “Great!”). In an embodiment, when the gaze returnsto the original position, the robot 500 may temporarily provide acertain facial expression (e.g., a happy facial expression 2720) for agiven time (e.g., about 1 second), and then provide the defaultexpression after the given time.

Example (C) shows that the accessory 2700 is mounted on the robot 500and is in a state of waiting for content execution. When there is nocontent execution by the accessory 2700 after the accessory 2700 ismounted, the robot 500 may determine that the content execution iswaiting, and thereby provide a feedback of a default facial expression2730.

FIG. 28 is a diagram illustrating an example of an accessory mounted ona robot and a related operation according to an embodiment of thedisclosure.

Referring to FIG. 28 , FIG. 28 shows an example in which an accessory2800 (e.g., a tablet) is mounted on the robot 500, and the robot 500provides a feedback in response to a user input using the accessory2800. In addition, FIG. 28 shows an example of an empathic response to auser's reaction to the content of the accessory 2800 throughinterworking between the robot 500 and the accessory 2800.

Example (A) shows that the accessory 2800 (e.g., a tablet) is mounted onthe robot 500 and starts to execute content (e.g., an intro process). Asshown, when the content is executed in response to a user input (e.g., atouch for playing the content) in the accessory 2800, the robot 500 mayprovide feedback such as a default facial expression 2810 and a relatedsound (e.g., “Follow me!”).

Examples (B), (C), and (D) show that the robot 500 provides an empathicresponse to the content executed through the accessory 2800 or areaction of the user using the content. For example, the robot 500 mayexpress an emotion linked to body playing content being executed in theaccessory 2800.

As shown in Examples (B), (C), and (D), the robot 500 may providevarious feedbacks associated with an empathic response to the executedcontent or a user's reaction. For example, the robot 500 may performreal-time sensing (e.g., context awareness) for a surroundingenvironment or user's expression or reaction and thereby provide, asrelevant feedback, changes in facial expressions 2810 and 2820 (e.g., asurprised or negative facial expression, an excited facial expression, ahappy facial expression, etc.) and their associated sounds.

According to an embodiment, the robot 500 may provide a related sound inaccordance with the progress of the content. For example, when the bodyplaying content (e.g., dance) is played, a related sound such as “jump”,“waddle”, “stomp”, “dance”, “sing”, or “swing” may be provided.

In an embodiment, the facial expression change may refer to a changefrom the default (e.g., neutral) facial expression 2810 to the otherfacial expression 2820 corresponding to each situation, or a change fromone facial expression (e.g., the surprised facial expression) accordingto the first response to another facial expression (e.g., the happyfacial expression) according to the second response. In a certainembodiment, the robot 500 may use a camera or a mounted accessory havinga camera function to perform the real-time sensing, and provide feedbackthrough image analysis.

In an embodiment, when the accessory 2800 (e.g., a function-connecteddevice) is mounted on the robot 500, the robot 500 may sense a reactionof the user (e.g., the child) using the content, and provide feedback toempathize with the user's response. For example, the robot 500 mayprovide an empathic response (e.g., a laughing or crying sound) suitablefor the content of the accessory 2800.

FIGS. 29 to 32 are diagrams illustrating examples of an accessorycombination mounted on a robot and a related operation according tovarious embodiments of the disclosure.

Referring to FIGS. 29 , to 32, various combinations of accessories ofdifferent types (or characteristics) may be mounted on the robot 500.Then, the robot 500 may identify the characteristics corresponding to acombination of accessories and then provide feedback by applyingfunctional properties corresponding to the identified characteristics.

Referring to FIG. 29 , FIG. 29 shows an example in which a firstaccessory 2910 of the first category (e.g., character type), a secondaccessory 2920 of the second category (e.g., event type), and a thirdaccessory 2930 of the second category are mounted in combination. In anembodiment, when the second and third accessories 2920 and 2930 aremounted in a state where the first accessory 2910 has been alreadymounted, the robot 500 may provide a facial expression and a sound ofthe characteristics (e.g., Halloween) of the second category, based onthe characteristics (e.g., bear) of the first category. For example,when the second accessory 2920 of a Halloween party hat and the thirdaccessory 2930 of a cloak are mounted after the first accessory 2910 ofbear's ears has been mounted, a bear character may express specificsound, motion and facial expression associated with the Halloween event.

Referring to FIG. 30 , FIG. 30 shows an example in which a firstaccessory 3010 of the first category (e.g., character type) and a secondaccessory 3020 (e.g., projector, tablet, etc.) of the third category(e.g., function-connected type) are mounted in combination. In anembodiment, when the second accessory 3020 is mounted in a state wherethe first accessory 3010 has been already mounted, the robot 500 mayprovide a function (e.g., video playback) through the second accessory3020 by applying the characteristics (e.g., interworking, adding orextending a function of the second accessory 3020) of the thirdcategory, based on the characteristics (e.g., bear) of the firstcategory. For example, when the second accessory 3020 of a tablet ismounted after the first accessory 3010 of bear's ears has been mounted,a video related to a bear character may be played in the tablet.

Referring to FIG. 31 , FIG. 31 shows an example in which a firstaccessory 3110 of the second category (e.g., event type) and a secondaccessory 3120 of the third category (e.g., function-connected type) aremounted in combination. In an embodiment, when the second accessory 3120is mounted in a state where the first accessory 3110 has been alreadymounted, the robot 500 may provide a function (e.g., video playback)through the second accessory 3120 by applying the characteristics (e.g.,interworking, adding or extending a function of the second accessory3120) of the third category, based on the characteristics (e.g.,Halloween) of the second category. For example, when the secondaccessory 3120 of a tablet is mounted after the first accessory 3110 ofa Halloween party hat has been mounted, a video related to Halloween maybe played in the tablet.

Referring to FIG. 32 , FIG. 32 shows an example in which a firstaccessory 3210 of the first category (e.g., character type), a secondaccessory 3220 of the second category (e.g., event type), and a thirdaccessory 3230 of the third category (e.g., function-connected type) aremounted in combination. In an embodiment, when the third accessory 3230is mounted in a state where the first and second accessories 3210 and3220 have been already mounted, the robot 500 may provide a function(e.g., video playback) through the third accessory 3230 by applying thecharacteristics (e.g., Halloween) of the second category, based on thecharacteristics (e.g., bear) of the first category. For example, whenthe third accessory 3230 of a tablet is mounted after the firstaccessory 3210 of bear's ears 2920 and the second accessory 3220 of aHalloween party hat have been mounted, a video related to bear andHalloween may be played in the tablet.

According to various embodiments, the robot 500 may provide a feedbackby considering environmental factors (e.g., a time, a time zone, aplace, a location) in addition to the type of a mounted accessory, thenumber of mounted accessories, and the position of the mountedaccessory.

In embodiments, depending on the time zone of mounting an accessory, therobot 500 may provide different feedback or functions.

In an embodiment, when the accessory is mounted during the daytime, therobot 500 may provide feedback based on functions available in thedaytime. For example, when a bear accessory is mounted, the robot 500may provide a sound of a size suitable for play. For example, when atablet is mounted, the robot 500 may provide brightness suitable forviewing in the daytime, or may preferentially provide content forlearning in the daytime. In an embodiment, when the accessory is mountedin the night time, the robot 500 may provide feedback based on functionsavailable in the night time. For example, when a bear accessory ismounted, the robot 500 may provide a quiet sound suitable for night timeor provide lighting adequate for the ambient light. For example, when atablet is mounted, the robot 500 may provide brightness suitable forviewing at night time, or may preferentially provide good content beforegoing to sleep at night.

In embodiments, depending on the place of mounting an accessory, therobot 500 may provide different feedback or functions.

In an embodiment, when the accessory is mounted in the living room, therobot 500 may provide functions available in the living room. Forexample, when a tablet is mounted, the robot 500 may provide contentsuitable for viewing with family and control the tablet to havebrightness suitable for indoor use. In an embodiment, when the accessoryis mounted in a child's room, the robot 500 may provide functionsavailable in the child's room. For example, when a tablet is mounted,the robot 500 may first provide content suitable for viewing with achild and control the tablet to have brightness according to thesurrounding brightness. In an embodiment, when the accessory is mountedoutdoors, the robot 500 may first provide content viewable outdoors andcontrol the brightness accordingly.

According to various embodiments of the disclosure, an operating methodof an electronic device may include detecting mounting of at least oneaccessory; identifying accessory characteristics associated with the atleast one accessory; determining properties of the electronic deviceassociated with the at least one accessory, based on the accessorycharacteristics; changing the properties of the electronic device, basedon the determined properties; and outputting at least one element of avisual element, an auditory element, or a tactile element associatedwith the at least one accessory, based on the changed properties.

According to various embodiments, the identifying accessorycharacteristics may include acquiring accessory information from the atleast one accessory when the mounting of the at least one accessory isdetected, and identifying the accessory characteristics, based on theaccessory information.

According to various embodiments, the identifying accessorycharacteristics may include identifying the accessory characteristics,based on at least one of a type of accessory, the number of mountedaccessories, or a mounting position of accessory, or a combinationthereof.

According to various embodiments, the outputting at least one elementmay include identifying the number of mounted accessories, and providinga feedback by realizing an intention corresponding to the identifiednumber of mounted accessories.

According to various embodiments, the determining properties of theelectronic device may include determining the properties of theelectronic device with different degrees of completion depending on theidentified number of mounted accessories.

According to various embodiments, the identifying accessorycharacteristics may include classifying the accessory characteristicsstage by stage depending on the identified number of mountedaccessories.

According to various embodiments, the outputting at least one elementmay include determining a modeling rule corresponding to the type ofaccessory, and providing a feedback based on the modeling rule stage bystage depending on the identified number of mounted accessories.

According to various embodiments, the outputting at least one elementmay include providing a feedback associated with the mounting of the atleast one accessory, based on the electronic device or the at least oneaccessory.

According to various embodiments, the outputting at least one elementmay include determining a modeling rule corresponding to the accessorycharacteristics, and providing a feedback as an immediate response to aninteraction with a user, based on the modeling rule.

According to various embodiments, the identifying accessorycharacteristics may include identifying the accessory characteristic byusing a learning model trained through an artificial intelligencealgorithm.

According to various embodiments of the disclosure, a user may mount anduse various accessories on the robot (or a robotic device or anelectronic device), and may change various functional characteristics(e.g., UX features and/or functions) of the robot according to the typeof accessory.

According to various embodiments of the disclosure, the robot maydetermine the characteristics of the accessory through a combination ofa type of accessory, a mounting position, and/or the number of mountedaccessories of the accessory, and may change and provide the functionalcharacteristics of the robot according to the characteristics of theaccessory.

According to various embodiments of the disclosure, the user may changethe UX features (e.g., display theme, face, facial expression, sound(e.g., voice), motion (or behavior, appearance), haptic, lightning,and/or color) of the robot and/or the functionality of the robot foreach the characteristics and stages of the accessory, rather than simplychanging an external appearance of the robot by mounting a device or anaccessory.

According to various embodiments of the disclosure, a specialized anddifferentiated service such as Family with Kids of a social robot may beprovided to a user. According to various embodiments of the disclosure,when providing the functional characteristics of the robot according tothe characteristics of the accessory, may provide an intelligent robotand intelligent service that may operate according to a situation (e.g.,a time and/or a time zone) and/or a place (e.g., a location and/or aspace) where the mounting of the accessory is made.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic device comprising: a housing; atleast one sensor circuit disposed in the housing and configured todetect mounting of at least one accessory; at least one communicationcircuit disposed in the housing and configured to communicate with theat least one accessory; a display device disposed in the housing andconfigured to display visual information associated with the at leastone accessory; an output device configured to output auditoryinformation associated with the at least one accessory; and at least oneprocessor disposed in the housing and operatively connected to thesensor circuit, the communication circuit, the display device, and theoutput device, wherein the at least one processor is configured to:detect the mounting of the at least one accessory, identify a type ofaccessory, the number of mounted accessories, and a mounting position ofthe accessory when the mounting of the at least one accessory isdetected, determine a modeling rule corresponding to the type ofaccessory, based on the modeling rule, change default properties of theelectronic device to specified properties of the electronic devicecorresponding to the modeling rule, provide change in feedback based onthe modeling rule stage by stage depending on the number of mountedaccessories and the mounting position of the accessory, and whenperforming an interaction with the user based on a user input, outputfeedbacks corresponding to the user input step by step through a visualelement, an auditory element, and a tactile element, based on thespecified properties.
 2. The electronic device of claim 1, wherein theat least one processor is further configured to: acquire accessoryinformation from the at least one accessory through the communicationcircuit when the mounting of the at least one accessory is detectedthrough the communication circuit, and identify the accessorycharacteristics, based on the accessory information.
 3. The electronicdevice of claim 1, wherein the at least one processor is furtherconfigured to: identify the number of mounted accessories, and providethe feedback by realizing an intention corresponding to the identifiednumber of mounted accessories.
 4. The electronic device of claim 1,wherein the at least one processor is further configured to: determinethe properties of the electronic device with different degrees ofcompletion depending on the identified number of mounted accessories. 5.The electronic device of claim 1, wherein the at least one processor isfurther configured to: classify the accessory characteristics stage bystage depending on the identified number of mounted accessories.
 6. Theelectronic device of claim 1, wherein the modeling rule is configuredto: designate a specific character assigned to the accessory, andprovide an output facial expression based on the specific character. 7.The electronic device of claim 6, wherein the modeling rule is furtherconfigured to provide a change in the facial expression stage by stage,depending on the number of mounted accessories.
 8. The electronic deviceof claim 1, wherein the at least one processor is further configured todetermine a path rule corresponding to the type of accessory, andwherein the path rule is configured to define a sequence of operationsof an application corresponding to the type of accessory.
 9. Theelectronic device of claim 1, wherein the at least one processor isfurther configured to: provide the feedback associated with the mountingof the at least one accessory, based on the electronic device or the atleast one accessory.
 10. The electronic device of claim 1, wherein theat least one processor is further configured to: determine a modelingrule corresponding to the accessory characteristics, and provide thefeedback as an immediate response to an interaction with a user, basedon the modeling rule.
 11. The electronic device of claim 1, wherein theat least one processor is further configured to: identify the accessorycharacteristic by using a learning model trained through an artificialintelligence algorithm.
 12. An operating method of an electronic device,the method comprising: detecting mounting of at least one accessory;identifying a type of accessory, the number of mounted accessories, anda mounting position of the accessory when the mounting of the at leastone accessory is detected; determining a modeling rule corresponding tothe type of accessory; based on the modeling rule, changing defaultproperties of the electronic device to specified properties of theelectronic device corresponding to the modeling rule; providing changein feedback based on the modeling rule stage by stage depending on thenumber of mounted accessories and the mounting position of theaccessory; and when performing an interaction with the user based on auser input, outputting feedbacks corresponding to the user input step bystep through a visual element, an auditory element, and a tactileelement, based on the specified properties.
 13. The method of claim 12,wherein the identifying of the accessory characteristics comprises:acquiring accessory information from the at least one accessory when themounting of the at least one accessory is detected; and identifying theaccessory characteristics, based on the accessory information.
 14. Themethod of claim 12, wherein the outputting of the at least one elementcomprises: identifying the number of mounted accessories; and providingthe feedback by realizing an intention corresponding to the identifiednumber of mounted accessories, and wherein the determining of theproperties of the electronic device comprises determining the propertiesof the electronic device with different degrees of completion dependingon the identified number of mounted accessories, and wherein theidentifying of the accessory characteristics comprises classifying theaccessory characteristics stage by stage depending on the identifiednumber of mounted accessories.
 15. The method of claim 12, wherein theoutputting of the at least one element comprises: providing the feedbackassociated with the mounting of the at least one accessory, based on theelectronic device or the at least one accessory.
 16. The method of claim12, wherein the outputting of the at least one element comprises:providing the feedback as an immediate response to an interaction with auser, based on the electronic device or the at least one accessory.